diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 1eea3f5..9e14428 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -1,67 +1,69 @@ -# Backlog +# Backlog & status -Open issues, seeding the Gitea tracker via `scripts/create-issues.sh` (run once the -repo exists). `[client]` = iOS, `[backend]` = the platform service on the Docker host, -`[ci]` = build/test. +Source-of-truth tracker. The Gitea issue tracker is currently **empty (0 issues)** — +run `scripts/create-issues.sh` with a token to populate it. `[client]` = iOS, +`[backend]` = the platform service on the Docker host, `[ci]` = build/test. + +Status: 🔴 open · 🟡 mitigated client-side (real fix still pending) · 🟢 done. ## Backend — data quality (highest priority) -1. **[backend] F1 stories are mis-classified into tech/finance/politics.** The topic - classifier has no sport category, so F1 (with 100% F1 sources) scatters across topics. - Evidence from the live feed: "Monaco Grand Prix stewards…" → `tech`, "F1 reports 35% - CO2 reduction…" → `finance`, "McLaren launch appeal against Gasly penalty" → `politics`. - Add a `sport` (or `f1`) topic and route motorsport sources there. -2. **[backend] Over-clustering merges unrelated articles.** Stories pair a headline with a - summary about something else, e.g. an Ebola/Bunia (DR Congo) headline carried a - South-Africa summary, and a CRISPR headline carried Drakensberg/SA text. Raise the - TF-IDF similarity threshold (~0.22 → 0.30–0.40) and make the canonical `headline` and - `summary` come from the **same** representative article. This also inflates - `sourceCount` / `crossSourceConfirmation` and the signal score. -3. **[backend] Add accurate per-story tags (topic + region).** So the client can *read* - tags (sport, uganda, east-africa, south-africa, …) instead of keyword-guessing. Unblocks - moving the filter pills to server-side. Pairs with #1/#2. -4. **[backend] Topic-classifier coverage for the 47 feeds.** Ensure science/world/tech map - sensibly into the supported topics. +1. 🟡 **[backend] F1 mis-classified into tech/finance/politics.** No sport category, so + F1 (100% F1 sources) scatters across topics: "Monaco Grand Prix…" → `tech`, + "F1 reports 35% CO2…" → `finance`, "McLaren launch appeal…" → `politics`. Add a + `sport`/`f1` topic and route motorsport sources there. + *Client mitigation shipped:* F1 pill matches clear F1 headlines + a majority of F1 + sources, and F1 is excluded from All/Tech/Finance. Backend still mis-tags. +2. 🔴 **[backend] Catastrophic over-clustering — unrelated articles merged on a single + shared word.** Newly confirmed on-device: a story merged a Russian-film-director + article + "Who's afraid of **Michael Jackson**?" + "Chelsea **Jackson** Roberts" + (wellness) — grouped on the surname *Jackson*. Another merged "Gunfire at Niger + **airport**" + "L.A. County Whiteman **Airport** crash" + "**Niger** fintech". Result: + wrong headline, summary, topic, and inflated signal. **Raise the TF-IDF threshold hard + (~0.22 → 0.45–0.55) and require multiple shared distinctive terms / named-entity + overlap, with a tight time window. If still over-merging, move to sentence embeddings + (all-MiniLM-L6-v2, cosine ≥ ~0.6).** Make canonical headline + summary come from the + same representative article. *No client fix is possible — the story object itself is + wrong.* +3. 🔴 **[backend] Add accurate per-story tags (topic + region).** So the client can read + tags (sport, uganda, south-africa, …) instead of keyword-guessing. Unblocks #6. +4. 🔴 **[backend] Topic-classifier coverage for the 47 feeds.** science/world/tech mapping. ## Backend — features -5. **[backend] Server-side push for breaking alerts (APNs or ntfy).** Needed for - breaking-story notifications while the app is fully closed; the backend already computes - the stories. Client currently does local notifications + background refresh only. +5. 🔴 **[backend] Server-side push for breaking alerts (APNs or ntfy).** For + breaking-story notifications while the app is closed. Client does local notifications + + background refresh only. ## Client -6. **[client] Move interest filters server-side once tags exist (#3).** Replace the - client-side keyword filters (F1 / Uganda / East Africa / South Africa) — which only see - loaded stories and match the headline — with backend tag filtering for full coverage. -7. **[client] SSID-based trusted networks.** Pin trusted Wi-Fi by name. Needs the Access - WiFi Information entitlement (real device + paid account); no-ops in the simulator. - Currently approximated by LAN reachability. -8. **[client] Optional in-app WireGuard tunnel.** NetworkExtension packet-tunnel so Jarvis - can bring up a WireGuard tunnel itself (WireGuard/Netmaker configs only). Entitlement + - real device. Deferred. -9. **[client] App icon + launch assets.** `AppIcon` is an empty placeholder. -10. **[client] Flesh out Latest / Search tabs.** (Saved is now an explicit bookmarks list.) - Add proper empty/loading states and dedicated behavior. -11. **[client] Verify hero image loading.** `AsyncImage` against real feed `imageUrl`s - (ATS, redirects, grey fallback) in the article reader. -12. **[client] Offline-first polish.** Pre-cache top stories for true offline-first; today - the green "cached" dot depends on opening an article. +6. 🔴 **[client] Move interest filters server-side once tags (#3) exist.** Today's pills + are client-side keyword/headline matches on loaded stories. *Refined since:* regions + match headline-only (no polluted summaries/outlets), regions exclusive, F1 by source + majority, East Africa removed. +7. 🔴 **[client] SSID-based trusted networks.** Needs Access WiFi Information entitlement + (device + paid account). Approximated by LAN reachability today. +8. 🔴 **[client] Optional in-app WireGuard tunnel.** NetworkExtension; entitlement + device. Deferred. +9. 🔴 **[client] App icon + launch assets.** `AppIcon` is an empty placeholder. +10. 🔴 **[client] Flesh out Latest / Search tabs.** Saved is now bookmarks; Latest/Search + still minimal. +11. 🔴 **[client] Verify hero image loading.** `AsyncImage` vs real feed `imageUrl`s. +12. 🔴 **[client] Offline-first polish.** Pre-cache top stories. ## CI -13. **[ci] Add unit/UI tests + a test matrix leg.** The build matrix compiles but runs no - tests yet. -14. **[ci] Backend CI build matrix.** Python 3.11–3.13 + lint + Docker image build/health - probe, for the backend repo (template lives in chat/handoff). +13. 🔴 **[ci] Add unit/UI tests + a test matrix leg.** Build matrix compiles, runs no tests. +14. 🔴 **[ci] Backend CI build matrix.** Python 3.11–3.13 + lint + Docker build/health probe. --- -## Recently shipped (this iteration) +## 🟢 Shipped -Signal feed + detail + reader + feed manager; LAN⇄Tailscale connectivity; notifications -(morning/evening briefings + breaking alerts) with Open-Meteo weather; device code signing; -feed resilience (no blank-on-refresh, ignore cancelled, coalesced WS refresh); clearer -empty states; deterministic feed order; read/unread (drop stripe on open); 6-line summary -preview; pinned mutually-exclusive filters (F1 split out, regions non-overlapping, -Finance/Tech by topic); swipe actions (left = save, right = share + mark read). +Signal feed + detail + reader + feed manager · LAN⇄Tailscale connectivity · notifications +(morning/evening briefings + breaking alerts) with Open-Meteo weather · device code signing +· feed resilience (no blank-on-refresh, ignore `cancelled`, coalesced WS refresh) · clearer +empty states (polling / unreachable+host / offline / caught-up) · deterministic feed order · +6-line summary preview · pinned filters (F1 split out, regions exclusive & headline-matched, +Finance/Tech by topic) · swipe actions (short→Save / long→Mark read, right→Share) · +read/seen handling (seen-tracking with per-load snapshot, read stories collapse into a +"Read" shelf with compact rows, "all caught up" state). diff --git a/scripts/create-issues.sh b/scripts/create-issues.sh index 64c5204..a14e1fb 100755 --- a/scripts/create-issues.sh +++ b/scripts/create-issues.sh @@ -25,8 +25,8 @@ mk() { # --- Backend: data quality --- mk "[backend] F1 stories mis-classified into tech/finance/politics" \ "The topic classifier has no sport category, so F1 (100% F1 sources) scatters across topics. Live evidence: 'Monaco Grand Prix stewards…' -> tech, 'F1 reports 35% CO2 reduction…' -> finance, 'McLaren launch appeal against Gasly penalty' -> politics. Add a sport/f1 topic and route motorsport sources there." -mk "[backend] Over-clustering merges unrelated articles" \ -"Stories pair a headline with an unrelated summary: an Ebola/Bunia (DR Congo) headline carried a South-Africa summary; a CRISPR headline carried Drakensberg/SA text. Raise the TF-IDF similarity threshold (~0.22 -> 0.30-0.40) and make canonical headline + summary come from the SAME representative article. Also inflates sourceCount/crossSourceConfirmation/score." +mk "[backend] Catastrophic over-clustering: unrelated articles merged on one shared word" \ +"Confirmed on-device: a story merged a Russian-film-director article + 'Who is afraid of Michael Jackson?' + 'Chelsea Jackson Roberts' (wellness), grouped on the surname Jackson. Another merged 'Gunfire at Niger airport' + 'L.A. County Whiteman Airport crash' + 'Niger fintech', grouped on airport/Niger. Wrong headline/summary/topic + inflated signal. Raise TF-IDF threshold hard (~0.22 -> 0.45-0.55), require multiple shared distinctive terms / named-entity overlap + tight time window; if still over-merging move to sentence embeddings (all-MiniLM-L6-v2, cosine >= 0.6). Canonical headline + summary must come from the SAME article. No client fix possible." mk "[backend] Add accurate per-story tags (topic + region)" \ "Expose accurate tags (sport, uganda, east-africa, south-africa, ...) so the client reads them instead of keyword-guessing. Unblocks server-side filter pills. Pairs with the F1 and over-clustering fixes." mk "[backend] Topic-classifier coverage for the 47 feeds" \