326d5fe983
clientRank already existed specifically to stop stale high-scorers from leading the feed, but computed age from updatedAt — the same timestamp that keeps resetting whenever the backend attaches any article to a cluster (BACKLOG #15). A fossil's age always read as ~0, silently defeating the whole mechanism. Fixed to use firstSeenAt. clientRank's decay is soft (max 20% penalty) and isn't always enough against a high signalScore alone, so added StorySummary.isFossil (>5 days old by firstSeenAt) as a hard backstop: mainStories and makeDigest now sink fossils below current stories, so one can never take the hero/lead slot regardless of score. Still shown in the feed, just not in the most prominent position. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
106 lines
6.9 KiB
Markdown
106 lines
6.9 KiB
Markdown
# Backlog & status
|
||
|
||
Source-of-truth tracker. The Gitea issue tracker is currently **empty (0 issues)** —
|
||
run `scripts/create-issues.sh` with a token to populate it. `[client]` = iOS,
|
||
`[backend]` = the platform service on the Docker host, `[ci]` = build/test.
|
||
|
||
Status: 🔴 open · 🟡 mitigated client-side (real fix still pending) · 🟢 done.
|
||
|
||
## Backend — data quality (highest priority)
|
||
|
||
1. 🟡 **[backend] F1 mis-classified into tech/finance/politics.** No sport category, so
|
||
F1 (100% F1 sources) scatters across topics: "Monaco Grand Prix…" → `tech`,
|
||
"F1 reports 35% CO2…" → `finance`, "McLaren launch appeal…" → `politics`. Add a
|
||
`sport`/`f1` topic and route motorsport sources there.
|
||
*Client mitigation shipped:* F1 pill matches clear F1 headlines + a majority of F1
|
||
sources, and F1 is excluded from All/Tech/Finance. Backend still mis-tags.
|
||
2. 🔴 **[backend] Catastrophic over-clustering — unrelated articles merged on a single
|
||
shared word.** Newly confirmed on-device: a story merged a Russian-film-director
|
||
article + "Who's afraid of **Michael Jackson**?" + "Chelsea **Jackson** Roberts"
|
||
(wellness) — grouped on the surname *Jackson*. Another merged "Gunfire at Niger
|
||
**airport**" + "L.A. County Whiteman **Airport** crash" + "**Niger** fintech". Result:
|
||
wrong headline, summary, topic, and inflated signal. **Raise the TF-IDF threshold hard
|
||
(~0.22 → 0.45–0.55) and require multiple shared distinctive terms / named-entity
|
||
overlap, with a tight time window. If still over-merging, move to sentence embeddings
|
||
(all-MiniLM-L6-v2, cosine ≥ ~0.6).** Make canonical headline + summary come from the
|
||
same representative article. *No client fix is possible — the story object itself is
|
||
wrong.*
|
||
3. 🔴 **[backend] Add accurate per-story tags (topic + region).** So the client can read
|
||
tags (sport, uganda, south-africa, …) instead of keyword-guessing. Unblocks #6.
|
||
4. 🔴 **[backend] Topic-classifier coverage for the 47 feeds.** science/world/tech mapping.
|
||
15. 🔴 **[backend] Freshness score decays from `lastUpdatedAt` (cluster last touched), not
|
||
from the actual news event age — old stories never leave the top of the feed.**
|
||
Confirmed live on `/api/v1/stories`: 13 of the top 20 stories are 5+ days old by
|
||
`firstSeenAt` but still score 90+; the #1 story (`story_511c67eb064d`) has
|
||
`firstSeenAt: 2026-06-17`, `lastUpdatedAt: 2026-07-09` (21 days later), `freshnessScore:
|
||
1.0`, signal 100. Compounds with #2 (over-clustering): a loose cluster keeps absorbing
|
||
tangential articles indefinitely (this story's timeline includes "Security forces ready
|
||
for June 30 marches: Ramaphosa" — unrelated to the football story it's attached to),
|
||
each attachment resets `lastUpdatedAt` to now, so the cluster's clock never runs out and
|
||
its canonical headline drifts to whatever was clustered last (list view showed "South
|
||
Africa stun South Korea…"; story detail showed "Updated FIFA rankings after Bafana
|
||
Bafana exit" for the *same* `story_id`). **Fix: decay freshness from `firstSeenAt` (or
|
||
the newest article whose relevance to the cluster core is verified), not from
|
||
"last touched."** Tightening the TF-IDF threshold in #2 should also reduce how often
|
||
unrelated articles keep a stale cluster's clock alive. Also confirmed the API's own
|
||
`isStale` field is unusable as a stopgap — it reads `false` on every story checked
|
||
regardless of actual age, so it isn't computed from a real staleness threshold.
|
||
*Client mitigations shipped:*
|
||
1. Age display (row meta line, hero card) now reads from `firstSeenAt` instead of
|
||
`updatedAt`, so the UI no longer shows "1 hr ago" on stories that are actually
|
||
weeks old.
|
||
2. Found that `StorySummary.clientRank` — an *existing* time-decay mechanism, already
|
||
in the codebase specifically to stop stale high-scorers from leading the feed — was
|
||
itself silently neutralized by this same bug: it computed age from `updatedAt`, so a
|
||
fossil's age always read as ~0. Fixed to use `firstSeenAt`. This isn't new
|
||
client-side ranking logic; it's un-breaking a pre-existing, intentional one.
|
||
3. Added `StorySummary.isFossil` (`firstSeenAt` > 5 days old) as a hard backstop —
|
||
`clientRank`'s decay is soft (max 20% score penalty) and isn't always enough to
|
||
overcome a high `signalScore` alone. Fossils are sunk below current stories in
|
||
`mainStories` and `makeDigest`, so a stale story can never occupy the hero/lead slot
|
||
even if the server still ranks it #1. They still appear in the feed, just not in
|
||
the most prominent, most misleading position.
|
||
*Still wrong at the source* — the server's own `signalScore`/`freshnessScore` ordering
|
||
(what page 2+, pagination cutoffs, and `min_signal` filtering all rely on) is
|
||
unaffected; these are display/ordering patches downstream of it. The real fix is
|
||
still backend-side.
|
||
|
||
## Backend — features
|
||
|
||
5. 🔴 **[backend] Server-side push for breaking alerts (APNs or ntfy).** For
|
||
breaking-story notifications while the app is closed. Client does local notifications +
|
||
background refresh only.
|
||
|
||
## Client
|
||
|
||
6. 🔴 **[client] Move interest filters server-side once tags (#3) exist.** Today's pills
|
||
are client-side keyword/headline matches on loaded stories. *Refined since:* regions
|
||
match headline-only (no polluted summaries/outlets), regions exclusive, F1 by source
|
||
majority, East Africa removed.
|
||
7. 🔴 **[client] SSID-based trusted networks.** Needs Access WiFi Information entitlement
|
||
(device + paid account). Approximated by LAN reachability today.
|
||
8. 🔴 **[client] Optional in-app WireGuard tunnel.** NetworkExtension; entitlement + device. Deferred.
|
||
9. 🔴 **[client] App icon + launch assets.** `AppIcon` is an empty placeholder.
|
||
10. 🔴 **[client] Flesh out Latest / Search tabs.** Saved is now bookmarks; Latest/Search
|
||
still minimal.
|
||
11. 🔴 **[client] Verify hero image loading.** `AsyncImage` vs real feed `imageUrl`s.
|
||
12. 🔴 **[client] Offline-first polish.** Pre-cache top stories.
|
||
|
||
## CI
|
||
|
||
13. 🔴 **[ci] Add unit/UI tests + a test matrix leg.** Build matrix compiles, runs no tests.
|
||
14. 🔴 **[ci] Backend CI build matrix.** Python 3.11–3.13 + lint + Docker build/health probe.
|
||
|
||
---
|
||
|
||
## 🟢 Shipped
|
||
|
||
Signal feed + detail + reader + feed manager · LAN⇄Tailscale connectivity · notifications
|
||
(morning/evening briefings + breaking alerts) with Open-Meteo weather · device code signing
|
||
· feed resilience (no blank-on-refresh, ignore `cancelled`, coalesced WS refresh) · clearer
|
||
empty states (polling / unreachable+host / offline / caught-up) · deterministic feed order ·
|
||
6-line summary preview · pinned filters (F1 split out, regions exclusive & headline-matched,
|
||
Finance/Tech by topic) · swipe actions (short→Save / long→Mark read, right→Share) ·
|
||
read/seen handling (seen-tracking with per-load snapshot, read stories collapse into a
|
||
"Read" shelf with compact rows, "all caught up" state).
|