Skip to content

Add a frequency tie-break and per-match frequencies to semsim search - #1484

Draft
kevinschaper wants to merge 1 commit into
feat/semsim-annotation-icfrom
feat/semsim-frequency-tiebreak
Draft

kevinschaper wants to merge 1 commit into
feat/semsim-annotation-icfrom
feat/semsim-frequency-tiebreak

Conversation

@kevinschaper

Copy link
Copy Markdown
Member

Closes #1478. Part of #1474. Based on #1483.

Summary

tie_break (none default, or frequency) and default_frequency (0.5) on search.

  • With frequency, entities with equal scores are ordered by how frequent the matched phenotypes are in each entity. For each query term, the harness takes the frequency of the entity annotation that best matches it, and sums those in log space.
  • Where frequency comes from:
    • OMIM has_quotient, such as 9 of 12 patients, when present.
    • Otherwise the midpoint of the HPO frequency category, such as "Occasional" giving 0.17.
    • Otherwise default_frequency.
    • Values are floored at 0.01, because some OMIM edges record 0 patients.
  • The tie-break runs inside the ranking query. So limit keeps the best of a tie group, not its first IDs.
  • Each best match in a ducksim search now carries frequency, the matched annotation's recorded frequency when it has one.

Design notes

  • Frequencies are computed per query. They come from a _freq view over edges, evaluated over only the entities that query touches, so nothing is computed at startup. The other option in semsim API: frequency tie-break for phenotype search #1478 was a normalised column built into the KG. That's still available if Solr or the UI need the numbers elsewhere.
  • Coverage. About 80% of disease-phenotype edges carry a frequency. Of human gene edges, 101,236 of 158,412 do. Model organism gene edges carry none, so there the tie-break has no effect.
  • A KG whose edges lack the frequency columns gets an empty view, and the tie-break falls back to ID order.

Testing

  • The backend suite passes: 793 passed, 16 skipped. The new fixture KG has four diseases tied on one phenotype, with frequencies of 0.9, "Occasional", none recorded, and 0 of 5. The tests cover ordering, default_frequency, the tie-break applying before the limit, per-match frequencies, KGs without frequency columns, and the API.
  • Checked against monarch-kg 2026-10-07, with full search and query-to-disease scoring. SLC35A2-CDG's position within the tie group:
Query Tie group Position with frequency tie-break
5 common terms + inverted nipples 22 11
5 common terms + hypsarrhythmia 52 10
5 common terms + both 5 3

https://claude.ai/code/session_01RmR3yV2ogn7Rx7bJnXXouU

`tie_break=frequency` orders entities with equal scores by how frequent the
matched phenotypes are in each entity: OMIM patient counts where recorded,
else the midpoint of the HPO frequency category, else `default_frequency`
(0.5). The tie-break runs inside the ranking query, so the limit keeps the
best of a tie group. Frequencies come from a view over the KG's edges,
evaluated per query; nothing is computed at startup.

Each best match in a ducksim search now carries the matched annotation's
recorded frequency, when it has one.

Part of #1478.

Claude-Session: https://claude.ai/code/session_01RmR3yV2ogn7Rx7bJnXXouU
@codecov

codecov Bot commented Oct 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.18033% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (feat/semsim-annotation-ic@d497868). Learn more about missing BASE report.

Files with missing lines Patch % Lines
backend/src/monarch_py/service/ducksim.py 97.36% 1 Missing ⚠️
Additional details and impacted files
@@                     Coverage Diff                      @@
##             feat/semsim-annotation-ic    #1484   +/-   ##
============================================================
  Coverage                             ?   85.77%           
============================================================
  Files                                ?      134           
  Lines                                ?     7911           
  Branches                             ?        0           
============================================================
  Hits                                 ?     6786           
  Misses                               ?     1125           
  Partials                             ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant