Tags: devYRPauli/looma
Tags
v2.1.5: fairer benchmark matching credits correct paraphrases
The token-overlap matcher in `looma benchmark` missed correct paraphrases whose
salient content was diluted by circumstantial words on both sides, double-
penalizing one correct extraction as both a false positive and a false negative
("memory leak from accumulated EventEmitter listeners" vs gold "event listeners
... accumulate ... memory leak"). Tokens are now morphologically normalized
(reusing the retrieval stemmer, which is left untouched) and a third match
clause credits a high-overlap paraphrase with an absolute floor of 3 shared
content tokens, so short coincidental overlaps still do not match.
Validated against labeled should-match / should-NOT-match pairs (test_benchmark).
The heuristic's score is unchanged at 0.86, confirming the change credits real
paraphrases rather than inflating every score.
Corrected eval picture: a local Qwen2.5-7B beats the heuristic at both common
quants (F1 0.90 at Q3, 0.95 at Q4_K_M, vs 0.86); LLM bug precision is 0.80
(heuristic-equal), not the 0.60 the old matcher reported. README updated.
134 tests (2 new).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
v2.1.4: drop dangling-preamble and injected-meta noise from weekly DE…
…CISIONS
Two surgical extraction guards, both routed through the shared filters so they
apply to the heuristic extractor, the weekly/today display layer, and the
optional local-LLM path:
- Generalize the meta guard's dangling-preamble rule from a fixed word-list
("... is:", "... done:") to the structural case: any line ending in
"<word>:" is a preamble announcing content on the next line, not a
self-contained memory ("Here's where we are and the decision:"). Mid-sentence
colons ("Decision: use gRPC over REST") are unaffected.
- Add the injected agent-memory-rule signature ("from a tool result this turn")
to the noise-phrase filter; it bleeds into turns untagged, so strip_injected
misses it and the architecture classifier read it as a design rule.
132 tests (2 new). Run `looma reprocess` to apply to an existing graph.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 7 + v1.0.0: hardening, health warnings, migrations; V1 reports - forward-compatible column migrations (pre-v1 DB upgrade safety) - graph-health degradation warnings (status --health) - V1_READINESS.md + LOOMA_V1_REPORT.md; README/CHANGELOG to v1.0.0 - version 1.0.0 (Development Status: Beta). 72 tests.
PreviousNext