kutesir/jarvis
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Update backlog statuses; strengthen over-clustering issue with new evidence
Some checks failed
CI / Build · macos-14 · Debug (push) Has been cancelled
Details
CI / Build · macos-15 · Debug (push) Has been cancelled
Details
CI / Build · macos-14 · Release (push) Has been cancelled
Details
CI / Build · macos-15 · Release (push) Has been cancelled
Details

Browse Source 
Mark each item open/mitigated/done, fold completed work into Shipped, and
upgrade the clustering bug with the on-device Jackson/airport over-merge proof
and a concrete TF-IDF/embeddings fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Robin Kutesa
2026-06-19 17:07:39 +03:00
parent 4331621ccc
commit 3d40065ec7
2 changed files with 54 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
scripts/create-issues.sh
View File

@@ -25,8 +25,8 @@ mk() {
# --- Backend: data quality ---
mk "[backend] F1 stories mis-classified into tech/finance/politics" \
"The topic classifier has no sport category, so F1 (100% F1 sources) scatters across topics. Live evidence: 'Monaco Grand Prix stewards…' -> tech, 'F1 reports 35% CO2 reduction…' -> finance, 'McLaren launch appeal against Gasly penalty' -> politics. Add a sport/f1 topic and route motorsport sources there."
mk "[backend] Over-clustering merges unrelated articles" \
"Stories pair a headline with an unrelated summary: an Ebola/Bunia (DR Congo) headline carried a South-Africa summary; a CRISPR headline carried Drakensberg/SA text. Raise the TF-IDF similarity threshold (~0.22 -> 0.30-0.40) and make canonical headline + summary come from the SAME representative article. Also inflates sourceCount/crossSourceConfirmation/score."
mk "[backend] Catastrophic over-clustering: unrelated articles merged on one shared word" \
"Confirmed on-device: a story merged a Russian-film-director article + 'Who is afraid of Michael Jackson?' + 'Chelsea Jackson Roberts' (wellness), grouped on the surname Jackson. Another merged 'Gunfire at Niger airport' + 'L.A. County Whiteman Airport crash' + 'Niger fintech', grouped on airport/Niger. Wrong headline/summary/topic + inflated signal. Raise TF-IDF threshold hard (~0.22 -> 0.45-0.55), require multiple shared distinctive terms / named-entity overlap + tight time window; if still over-merging move to sentence embeddings (all-MiniLM-L6-v2, cosine >= 0.6). Canonical headline + summary must come from the SAME article. No client fix possible."
mk "[backend] Add accurate per-story tags (topic + region)" \
"Expose accurate tags (sport, uganda, east-africa, south-africa, ...) so the client reads them instead of keyword-guessing. Unblocks server-side filter pills. Pairs with the F1 and over-clustering fixes."
mk "[backend] Topic-classifier coverage for the 47 feeds" \