Skip to content

Absolute shared-fingerprint floor alongside Jaccard thresholdΒ #27

Description

@Dominik-O22

Problem

matchingPairs in src/TypeScriptDuplicateFinder.ts gates pairs purely on the Jaccard similarity(left, right) >= threshold. A small candidate can clear a high ratio on very few shared fingerprints β€” e.g. two 6-node candidates sharing 5 fingerprints hit ~0.83 Jaccard but represent almost no shared structure. --min-nodes (default 20) mitigates this by raising the candidate-size bar, but ratio-only matching still admits high-ratio/low-absolute-overlap pairs near the size floor.

Proposed change

Add an absolute shared-fingerprint floor applied in addition to the Jaccard threshold:

  • New option --min-shared-fingerprints <n> (parsed in src/Options.ts), default off / 0.
  • In the pair gate (src/TypeScriptDuplicateFinder.ts, the score >= threshold site, ~L100), also require the intersection size to be at least the floor. The intersection count is already computed inside similarity (shared); expose it or inline the check so no second pass is needed.

Acceptance

  • Two tiny same-shape candidates that pass Jaccard but share fewer than the floor are dropped.
  • A large genuine duplicate (high absolute overlap) is unaffected.
  • Flag off -> behavior byte-for-byte unchanged.

Related

False-positive-reduction set (#19-#24); codex-surfaced. Complements the kind-diversity floor (#20): that bounds intra-candidate uniformity, this bounds inter-candidate overlap magnitude.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions