All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- ScientificReport schema 1.3.0 — immutable SourceRecord, claim–evidence graph, write barrier, reconstruct path (current public artifact baseline)
- ScientificReport schema 1.2.0 — lifecycle fields,
evidence_content_hash,validation_run_hash(still accepted; unchanged by decision-time companions) - ScientificReport schema 1.1.0 — Ed25519 signature, review chain hash, audit metadata
- ScientificReport schema 1.0.0 — initial canonical artifact model
- Companion schemas 1.0.0 —
integration_envelope,decision_time_evidence_bundle,akta_action_context,scientific_outcome_link(additive; do not mutate ScientificReport)
- ADR 0001: VSA / AKTA / Scientific Memory boundaries and decision-time semantics
SECURITY.mdanddocs/threat-model.md- Integration envelope builder (
src/vsa/identity/) - Decision-time evidence bundle with cutoff filter, verify, and compare (
src/vsa/decision_time/) - AKTA action-context export with non-authorization semantic validation (
src/vsa/export/akta.py) - PCS-shaped portable evidence refs and signed bundle verification
- Pilot modes (retrospective / shadow / read_only), privacy-safe metrics, outcome append links
- CLI:
snapshot-evidence,verify-decision-time-bundle,compare-cutoffs,export-akta-context,link-outcome - Operator guide, decision-time quick start, and independent reconstruction commands:
docs/operator-decision-time.md,docs/decision-time-quickstart.md - Public companion schema mirrors under
schemas/(byte-identical tosrc/vsa/schemas/) - Adversarial decision-time tests (tamper, forged signature, digest mismatch, missing portable refs)
- Dev quality tooling: Ruff (warnings as errors) and MyPy scoped to companion modules
- Contradiction/uncertainty hardening for decision-time bundles (post-cutoff, high-quality conflicts, metadata-only, entity ambiguity, computational→experimental extrapolation)
- ScientificReport advisory checks now emit
warnstatus (stale evidence, entity ambiguity, computational→experimental language) - Bundle verify fail-closed on missing portable refs, corrupted non-claims; signatures verified over recomputed hash
privacy_safe_metricsincluded inbundle_hash; AKTA export rejects evidence digest disagreement with report- Missing timestamps fail closed in all snapshot modes (not only retrospective/read_only)
- Verify fail-closed on included/excluded evidence_id overlap and duplicate included IDs
- Privacy-safe metrics revalidated on verify (reject digest/secret-shaped keys)
- Optional out-of-band
--trusted-keypin for decision-time signature verify compare-cutoffs --fail-on-verifyand clearer CLI load errors (missing/empty/invalid JSON)- AKTA rejects grant-like
tool_or_execution_classtokens
- README decision-time CLI section, claim boundaries, upstream PCS/AKTA/signing pins
- Docs index versions table includes companion schemas 1.0.0
- Schema docs: public mirrors (not symlinks); ClinVar ambiguity → low reliability
- Scientific credibility module (
src/vsa/scientific/credibility.py) with ambiguity caps and report warnings - Validation checks: ambiguous reliability cap, AlphaFold predicted labeling, ClinVar ambiguity visibility
- Benchmark expanded to 50 tasks with category minimums (10 adversarial, 5 each for ambiguity/contradiction/metadata-only/no-evidence)
- Core benchmark metrics: source recall/precision, citation integrity, evidence-ID validity, review-boundary accuracy, contradiction detection, bundle reproducibility
- Tests for credibility policies and Materials Project missing-key degradation
- ClinVar ambiguous queries always return
reliability: low - UniProt gene search sets
retrieval_ambiguityand caps reliability to low - Materials Project skipped cleanly with explicit warning when API key missing
- Scientific credibility warnings in markdown and HTML render output
- Retrieval pipeline applies credibility hardening before evidence is returned
- Acceptance bar and CI assert benchmark category minimums (50 tasks)
- Streamlit UI surfaces scientific credibility warnings and evidence reliability/ambiguity flags
vsa benchmarkprints category coverage summary on success
- Optional API key auth (
VSA_API_KEY,X-API-Keyheader) vsa verify-reviewtop-level commandscripts/acceptance.shandmake acceptance(CI parity)- Mocked connector tests: Semantic Scholar, Europe PMC
- Release workflow attaches full artifact bundle zip
- Semantic Scholar connector sets
content_levelmetadata - Build pipeline adds metadata-only CONTENT WARNING to limitations
- Acceptance CI job uses shared acceptance script
- Documentation consolidated and updated across README, docs/, and RELEASE_STATUS
- Review subcommands:
start,approve-claim,request-corrections,reject,verify - API review endpoints:
/v1/review/start,/v1/review/approve-claim,/v1/review/verify - Typed Pydantic request/response models for core API routes
- Structured API error codes (
MISSING_INPUT,VALIDATION_ERROR,RATE_LIMITED) - Optional rate limiting (
VSA_API_RATE_LIMIT) and deterministic mode (VSA_API_DETERMINISTIC) - Review chain verification via
vsa review verifyandverify_review_chain()
docs/api.mdexpanded with env vars, error format, review parity- CI runs review workflow smoke after bundle verification
vsa verify-bundle— manifest hash verification and attestation check- Export bundle v1.1:
report.md,sources/, manifestcreated_at, schema/validation versions - Benchmark regression gate (100% pass rate), expanded metrics, 27 tasks (fake evidence ID, stale source)
- CI acceptance job: clean-clone
make demo && make test && vsa benchmark docs/benchmark.md; RELEASE_STATUS verified-commit table
- Deeper variant/paper/protein/material claim templates
- ClinVar HGVS-only lookup; UniProt gene disambiguation warnings
- macOS CI smoke installs
[signing]; Makefileinstallincludes[api] - Metadata-only paper claims use explicit CONTENT WARNING
- Publication content levels (
metadata,abstract,fulltext) on OpenAlex, Crossref, and Europe PMC connectors - Paper claim templates: bibliographic identity, abstract-derived observation, full-text availability flag
- Export bundle manifest.json with SHA-256 hashes for all artifacts; attestation included by default
vsa compare-auditfor diffing audit artifacts across runs- Validation check: warn when all publication evidence is metadata-only
- Streamlit: export bundle download and attestation generation in Provenance tab
- E2E pipeline test (build → validate → audit → attest → export)
- Expanded benchmark gold labels (paper_doi, tp53, egfr)
- REST API (
vsa serve,[api]extra):/v1/build,/validate,/audit,/attest,/retrieve,/health - SLSA Provenance v1 / in-toto Statement attestation (
vsa attest,vsa verify-attestation) - OpenTelemetry hooks (
VSA_OTEL_ENABLED=1,[otel]extra) with httpx instrumentation - Schema migration CLI (
vsa migrate-schema) for 1.0.0/1.1.0 → 1.2.0 - Benchmark gold label scoring (
gold_labelsin tasks.json) - Live connector tests (
pytest -m live) and weekly GitHub Actions workflow - macOS CI smoke job
- Schema 1.2.0 lifecycle fields:
input_question,retrieval_plan,retrieval_warnings,limitations,domain, etc. - Provenance:
evidence_content_hash,validation_run_hash - ClinVar candidate ranking (up to 10 search hits, ambiguity flags, reliability scoring)
- Paper deduplication across OpenAlex/Crossref/PubMed/Europe PMC/Semantic Scholar
- UniProt Swiss-Prot vs TrEMBL labeling; AlphaFold predicted-structure warnings
- 25-task offline benchmark with adversarial cases and expanded scoring
- Audit artifacts:
vsa audit --out,vsa export --out-dir - Docs:
docs/architecture.md,docs/schema.md,docs/connectors.md,docs/release_checklist.md RELEASE_STATUS.md, README badges, known limitations- Connector tests under
tests/connectors/ - Expanded variant claim templates (identity, classification, ambiguity)
- CI uploads report/audit/benchmark artifacts
make demouses--audit-mode rulefor deterministic output
- LLM-backed scientific verifier (
vsa audit --audit-mode auto|rule|llm) - Hybrid audit merge: rule structural checks plus LLM semantic review (conservative)
- Audit output fields:
missing_evidence,confidence_concerns,verifier_method - Packaged verifier prompt at
src/vsa/prompts/verifier_prompt_v1.md - Streamlit audit mode selector (auto / rule / llm)
- Ed25519 report signing (
vsa sign,vsa verify-signature) via[signing]extra - Rule-based scientific audit layer (
vsa audit) implementing verifier prompt logic - Schema 1.1.0:
signature,review_chain_hash,review_event_hash, PDF output format - Legacy claim-ledger migration (
vsa migrate) - Benchmark CLI (
vsa benchmark) with offline and--livemodes - Materials domain example input and pipeline test
- Streamlit: audit, sign, and verify-signature actions in Provenance tab
human_reviewincluded in provenancereport_hashcorehuman_review.requiredis conditional (clinical/speculative/contradictions), not always true- Provenance preserves
signature,generated_by, andreview_chain_hashacross re-stamping
- NCBI PubMed connector (
src/vsa/connectors/ncbi_pubmed.py) - PDF report renderer (
pip install verified-science-agent[pdf]) - Evidence graph (Mermaid) in UI and
src/vsa/render/graph.py - Streamlit v3: build pipeline, LLM controls, human review tab, PDF download
- Material/chemical subject parsing (LiFePO4, cathode keywords)
- Retrieval warnings surfaced in CLI and UI
- Release workflow on git tags (
.github/workflows/release.yml)
- Connector cache now propagates through
--cache-dir - Materials Project routing for material entity types
- CLI:
inspect --json,compare --strict,review --out --notes - Legacy scripts delegate to
vsaCLI
- LLM claim extraction via OpenAI and Anthropic (
--claim-mode auto|rule|llm) vsa extractcommand for inspectable claim JSONvsa reviewcommand for claim-by-claim human review workflow- Materials Project connector (requires
MATERIALS_PROJECT_API_KEY) - Environment config via
.env(python-dotenv) .env.exampletemplate
vsa buildauto-selects LLM when API keys are present; falls back to rule-based- Provenance records review events and extraction method stack
0.1.0 - 2026-05-30
- Canonical
ScientificReportJSON Schema (v1.0.0) with support for genomics, proteins, papers, chemicals, materials, and experiments - Installable Python package with
vsaCLI:validate,render,hash,inspect,compare,retrieve,build - Validation engine with schema conformance, evidence linkage, confidence bounds, review boundaries, contradiction detection, and provenance hash verification
- Read-only database connectors: OpenAlex, Europe PMC, UniProt, ClinVar, AlphaFold DB, Crossref, Semantic Scholar
- Evidence retrieval pipeline with file-based caching
- Rule-based claim extraction (no LLM source field invention)
- Explainable evidence quality scoring
- Markdown and HTML report renderers
- Streamlit UI v2 with claim explorer, evidence view, provenance, and validation failures
- Benchmark suite with 6 domain tasks and offline fixtures
- CI workflow: schema validation, pytest, CLI smoke tests, render/hash tests
- Example reports and bad examples for regression testing