Neural Knowledge Runtime

Experimental offline QA system. Knowledge is stored in local packs; a small extractive reader should answer only from retrieved source passages.

question → local retrieval → source passages → extractive reader → answer + evidence

Current implementation is Python 3.12, English-only, and CPU-only.

src/neural_knowledge_runtime/
  packs/       Wikipedia wikitext importer and NKP serialization
  retrieval/   text normalization, BM25 indexing, search
  benchmarks/  benchmark generation and retrieval evaluation
  training/    custom reader, tokenizer, CPU trainer, QA evaluation
  cli.py       nkr command
benchmarks/    source fixtures and generated datasets
models/        local experimental checkpoints
scripts/       reproducible dataset builders
TODO.md        status and remaining work

nkr build converts saved Wikipedia wikitext into JSON .nkp files. NKP v2 stores source URL and SHA-256, article metadata, prose chunks, infobox facts, simple table rows, and a persisted JSON BM25 index.

uv run nkr build article.wiki --output article.nkp \
  --title "Article title" \
  --source-url "https://en.wikipedia.org/wiki/Article_title"

The importer is intentionally limited. It is not a complete MediaWiki parser. The index is readable JSON, not the final compact storage format.

nkr search runs local BM25 over a supplied pack. It case-folds, removes diacritics, removes basic question stop words, applies light English stemming, and boosts matching title, section, and fact records.

uv run nkr search article.nkp "When was the article subject born?"

There is no multi-article catalog or global pack search yet.

benchmarks/wikipedia_2026_08_01 is a reproducible pack-construction fixture. Its cloze questions are plumbing tests, not quality claims.

benchmarks/squad_v1_1_5x50 has 250 human-authored SQuAD v1.1 questions across five Wikipedia-derived articles. Retrieval-only baseline:

Recall@1: 63.2%
Recall@3: 77.6%
Recall@5: 83.2%
MRR@5:    0.709

uv run nkr eval benchmarks/squad_v1_1_5x50/generated

The reader is NKR-owned, not an external QA wrapper. It is a CPU-only four-layer Transformer (hidden size 192, four heads) with start-token, end-token, and no-answer heads. The current checkpoint has about 2.22M parameters because its learned vocabulary is small.

Training dependencies are separate from the runtime:

uv run --group train nkr train-reader \
  benchmarks/squad_v1_1_5x50/generated \
  --output models/nkr-reader-v1.pt \
  --epochs 10

The trainer uses a deterministic 80/20 question split and reports held-out oracle F1 samples. Full reader evaluation:

uv run --group train nkr eval-reader \
  benchmarks/squad_v1_1_5x50/generated \
  --checkpoint models/nkr-reader-v1.pt \
  --split validation

Use a checkpoint against one pack:

uv run --group train nkr ask article.nkp "Question" \
  --checkpoint models/nkr-reader-v1.pt

Reader quality is currently unproven: there is no language pretraining, the tokenizer is basic, and no complete held-out QA score has been recorded.

uv run python -m unittest discover -s tests -v
uv run --group train python -m unittest discover -s tests -v

This is a working prototype of pack construction, retrieval, benchmarks, and a first custom CPU reader. It is not a finished offline QA runtime.

I simply do not have time to continue this project beyond what it is right now. I will revisit it later.