From fcd4e2cce5ca4b6babcfaf7875929e0adb75186f Mon Sep 17 00:00:00 2001 From: Kutesir Date: Fri, 10 Jul 2026 03:17:25 +0300 Subject: [PATCH] =?UTF-8?q?docs:=20log=20freshness-decay=20bug=20=E2=80=94?= =?UTF-8?q?=20old=20stories=20never=20leave=20the=20top=20of=20the=20feed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Live-verified against the backend API: 13/20 top stories are 5+ days old by firstSeenAt but still score 90+. Freshness decays from lastUpdatedAt (cluster last touched) instead of the actual news event age, and it compounds with the known over-clustering bug (#2) β€” tangential articles keep attaching to old clusters, resetting their clock indefinitely. Co-Authored-By: Claude Sonnet 4.6 --- docs/BACKLOG.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 9e14428..b883403 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -28,6 +28,22 @@ Status: πŸ”΄ open Β· 🟑 mitigated client-side (real fix still pending) Β· 🟒 3. πŸ”΄ **[backend] Add accurate per-story tags (topic + region).** So the client can read tags (sport, uganda, south-africa, …) instead of keyword-guessing. Unblocks #6. 4. πŸ”΄ **[backend] Topic-classifier coverage for the 47 feeds.** science/world/tech mapping. +15. πŸ”΄ **[backend] Freshness score decays from `lastUpdatedAt` (cluster last touched), not + from the actual news event age β€” old stories never leave the top of the feed.** + Confirmed live on `/api/v1/stories`: 13 of the top 20 stories are 5+ days old by + `firstSeenAt` but still score 90+; the #1 story (`story_511c67eb064d`) has + `firstSeenAt: 2026-06-17`, `lastUpdatedAt: 2026-07-09` (21 days later), `freshnessScore: + 1.0`, signal 100. Compounds with #2 (over-clustering): a loose cluster keeps absorbing + tangential articles indefinitely (this story's timeline includes "Security forces ready + for June 30 marches: Ramaphosa" β€” unrelated to the football story it's attached to), + each attachment resets `lastUpdatedAt` to now, so the cluster's clock never runs out and + its canonical headline drifts to whatever was clustered last (list view showed "South + Africa stun South Korea…"; story detail showed "Updated FIFA rankings after Bafana + Bafana exit" for the *same* `story_id`). **Fix: decay freshness from `firstSeenAt` (or + the newest article whose relevance to the cluster core is verified), not from + "last touched."** Tightening the TF-IDF threshold in #2 should also reduce how often + unrelated articles keep a stale cluster's clock alive. *No client fix possible β€” the + story object's freshness field itself is wrong before it reaches the app.* ## Backend β€” features