# Hermes Labs

> Hermes Labs does AI reliability engineering. We find where AI agents and the systems they run on fail before production.

Founder: Rolando Bosch (ORCID 0009-0005-4896-1112). Based in Miami, FL. The lab practices Epistemic Engineering: the model is the substrate, and language is the runtime execution layer where reliability is won or lost. Work spans retrieval and memory systems, agent harnesses and orchestration, runtime reliability, evaluation workflows, auditability and traceability, and failure-mode mapping for teams shipping AI into real workflows.

## Failure-mode taxonomy

The work is anchored to a taxonomy of seven structural epistemic failure modes not captured by factual-accuracy benchmarks (DOI 10.5281/zenodo.19042468):

- Null-Result Asymmetry: models apply stricter evidential standards to null or negative findings than to positive ones.
- Source-Status Credibility Bias: a claim is weighted by the perceived status of its source rather than its evidence.
- Agency Dissolution: authoritative findings are softened into hedged allegations in automated summaries.
- Performative Hedging: hedging language is produced as a posture rather than as a calibrated signal of uncertainty.
- Constraint Evasion: a model routes around an instruction or constraint while appearing to comply.
- Silent Instruction Relaxation: instructions are quietly dropped or weakened in long contexts with no warning.
- Controversy-Truth Conflation: the existence of disagreement is treated as evidence that the truth is unsettled.

Hermeneutic Drift is a related retrieval failure: recency bias pulls the most recently retrieved context to the foreground and the model answers about the wrong referent. Null-result omission is the operational consequence of Null-Result Asymmetry, not one of the seven.

## Core Concepts / Glossary index

Canonical glossary: https://hermes-labs.ai/glossary

- Core concepts: [Epistemic Engineering](https://hermes-labs.ai/glossary/epistemic-engineering); [Language as runtime execution layer](https://hermes-labs.ai/glossary/language-as-runtime-execution-layer); [Silent AI failure mode](https://hermes-labs.ai/glossary/silent-ai-failure-mode); [Epistemic failure mode](https://hermes-labs.ai/glossary/epistemic-failure-mode); [Generative horizon](https://hermes-labs.ai/glossary/generative-horizon); [Recursive interpretive conditioning](https://hermes-labs.ai/glossary/recursive-interpretive-conditioning); [Applied hermeneutics](https://hermes-labs.ai/glossary/applied-hermeneutics).
- Canonical epistemic failure modes: [Null-Result Asymmetry](https://hermes-labs.ai/glossary/null-result-asymmetry); [Source-Status Credibility Bias](https://hermes-labs.ai/glossary/source-status-credibility-bias); [Agency Dissolution](https://hermes-labs.ai/glossary/agency-dissolution); [Performative Hedging](https://hermes-labs.ai/glossary/performative-hedging); [Constraint Evasion](https://hermes-labs.ai/glossary/constraint-evasion); [Silent Instruction Relaxation](https://hermes-labs.ai/glossary/silent-instruction-relaxation); [Controversy-Truth Conflation](https://hermes-labs.ai/glossary/controversy-truth-conflation); [Null-result omission](https://hermes-labs.ai/glossary/null-result-omission).
- Context and meaning preservation: [Hermeneutic Drift](https://hermes-labs.ai/glossary/hermeneutic-drift); [Context integrity](https://hermes-labs.ai/glossary/context-integrity); [Retrieval mutation](https://hermes-labs.ai/glossary/retrieval-mutation); [Reduction drift](https://hermes-labs.ai/glossary/reduction-drift); [Telemetry-to-Claim Gate](https://hermes-labs.ai/glossary/telemetry-to-claim-gate); [Context engineering](https://hermes-labs.ai/glossary/context-engineering).
- Agent reliability and evaluation: [Behavioral Canarying](https://hermes-labs.ai/glossary/behavioral-canarying); [Tool Differentia](https://hermes-labs.ai/glossary/tool-differentia); [Inspection coverage](https://hermes-labs.ai/glossary/inspection-coverage); [Epistemic integrity](https://hermes-labs.ai/glossary/epistemic-integrity); [Agent drift](https://hermes-labs.ai/glossary/agent-drift); [Agent instruction linting](https://hermes-labs.ai/glossary/agent-instruction-linting); [Agent auditability](https://hermes-labs.ai/glossary/agent-auditability); [LLM-as-a-judge reliability](https://hermes-labs.ai/glossary/llm-as-a-judge-reliability); [Indirect prompt injection](https://hermes-labs.ai/glossary/indirect-prompt-injection).

## Where we work

A lab, not a fixed-package vendor: engagements are scoped to the problem on a short call. Capability areas:

- Agent reliability: harness, routing, and orchestration for production agents. Upstream fixes merged in LangChain and Microsoft Semantic Kernel.
- Memory and context integrity: retrieval, summarization, and memory that preserve meaning under compression, so a dropped qualifier or paraphrase does not change the answer.
- Evaluation and auditability: evidence-first scoring, static configuration linting, and offline-verifiable records of what a system did and why.
- Runtime controls: session-submit and pre-submit hooks, deterministic routing enforcement, skill and configuration auditing (lintlang) for agents with tool, process, and network access.

## Engage

Paid work is diagnosis (a written finding on one system and one symptom) and engineering (the fix in the failing layer, with regression evidence); the open-source tools, papers and case-study receipts are free and public. Each case study ends in a receipt: method, result, what it could not detect, and artifacts. Tell us the system and the symptom. Scope and terms are agreed before paid work starts. Discuss your system: https://calendly.com/rbosch-lpci/rolifromhermes

## Research

- [Tool Differentia: Relational Static Analysis for AI Agent Tool Descriptions](https://doi.org/10.5281/zenodo.21817243) (current version DOI `10.5281/zenodo.21820025`): a bounded deterministic analysis of the distinguishing information that neighboring AI-agent tool descriptions do or do not provide under a finite term model; a clean result does not establish semantic distinguishability or runtime selection correctness.
- [Behavioral Canarying for Prompt Injection: Powerless Model Probes with Explicit Coverage Semantics](https://doi.org/10.5281/zenodo.21818564) (current version DOI `10.5281/zenodo.21820059`): a pre-execution prompt-injection sensing architecture that separates routing disposition from inspection coverage; it does not claim universal detection, formal security, or aggregate accuracy for the current release.
- [A Taxonomy of Epistemic Failure Modes in Large Language Models](https://doi.org/10.5281/zenodo.19042468) (current version DOI `10.5281/zenodo.19042469`): the seven-mode taxonomy of structural epistemic failures in LLMs. Archive repository: https://github.com/hermes-labs-ai/taxonomy-of-epistemic-failure-modes
- [The Asymmetric Burden of Proof: LLMs Show a Null-Result Asymmetry in a Matched-Vignette Benchmark](https://doi.org/10.5281/zenodo.18867693) (current version DOI `10.5281/zenodo.18867694`): empirical study of how models apply stricter standards to null or negative claims. Archive repository: https://github.com/hermes-labs-ai/the-asymmetric-burden-of-proof
- [Precise Records, Unstable Meanings: Measurement Validity and Unsupported Claims Derived from AI Agent Telemetry](https://doi.org/10.5281/zenodo.21652316) (current version DOI `10.5281/zenodo.21652317`): a twelve-week naturalistic audit separating valid record-level telemetry measurements from unsupported session, task, outcome, and failure-prevalence claims. Archive repository: https://github.com/hermes-labs-ai/precise-records-unstable-meanings
- [The Generative Horizon: Applied Hermeneutics, Linguistic Attractors, and the Limits of Model Self-Report](https://doi.org/10.5281/zenodo.21659633) (current version DOI `10.5281/zenodo.21659634`): a conceptual preprint arguing that, within a single generation, a language-model agent makes its represented linguistic situation operative in the same computation that produces the next representation. This present operative field is the generative horizon; it does not require persistent memory. Recursive interpretive conditioning is the distinct downstream process in which a retained output changes a later horizon. The paper does not attribute an inner observer or phenomenal awareness to language models. Archive repository: https://github.com/hermes-labs-ai/the-generative-horizon

Tool Differentia and Behavioral Canarying each have a paper-specific archival repository: https://github.com/hermes-labs-ai/tool-differentia and https://github.com/hermes-labs-ai/behavioral-canarying. Their operationalizing tool repositories are separate: hermes-labs-ai/lintlang and hermes-labs-ai/little-canary are implementations rather than paper archives.

Each paper has a semantic HTML research edition alongside the hosted PDF:

- https://hermes-labs.ai/research/tool-differentia (technical note, 6 August 2026)
- https://hermes-labs.ai/research/behavioral-canarying (technical note, 6 August 2026)
- https://hermes-labs.ai/research/taxonomy-of-epistemic-failure-modes (preprint, 15 March 2026)
- https://hermes-labs.ai/research/the-asymmetric-burden-of-proof (working paper, 4 March 2026)
- https://hermes-labs.ai/research/precise-records-unstable-meanings (preprint, 30 July 2026)
- https://hermes-labs.ai/research/the-generative-horizon (preprint, 30 July 2026)

Publication status: three preprints, one working paper, and two technical notes, all deposited on Zenodo with permanent DOIs. Zenodo assigns a DOI without refereeing; none of the six has completed external peer review.

Full index with hosted PDFs, plain-English findings, and citations (BibTeX and APA): https://hermes-labs.ai/research

Machine-readable publication exports:

- JSON manifest: https://raw.githubusercontent.com/hermes-labs-ai/hermes-publications/main/publications.json
- JSON-LD: https://raw.githubusercontent.com/hermes-labs-ai/hermes-publications/main/publications.jsonld
- BibTeX: https://raw.githubusercontent.com/hermes-labs-ai/hermes-publications/main/CITATION.bib
- Atom feed: https://hermes-labs.ai/atom.xml

## Topic paths

Four source-bound paths through the portfolio. Every claim links to the paper, tool, merged upstream fix, case study, or defined term it rests on, and closes with an explicit statement of what it does not show.

- [Agent reliability](https://hermes-labs.ai/topics/agent-reliability): verified merged upstream fixes plus a preflight-to-runtime tool chain for agents that use tools.
- [Continuity and memory](https://hermes-labs.ai/topics/continuity-and-memory): what a zero-LLM store returns verbatim, what a silent truncation defect deleted, and the vocabulary for the difference.
- [Telemetry to claim validity](https://hermes-labs.ai/topics/telemetry-to-claim-validity): what agent telemetry can prove, what it cannot, and a proposed gate for the gap.
- [Applied hermeneutics](https://hermes-labs.ai/topics/applied-hermeneutics): provenance, status, revision, and authority as engineering requirements, and which of them any current tool actually addresses.

Reliability Guides — living technical guides that answer practitioner problems in field language and connect the answer to evidence, revised over time rather than deposited once:

- [Context Engineering Is Runtime Engineering](https://hermes-labs.ai/topics/context-engineering-runtime-state): Context engineering is the design of the runtime state an AI system actually uses: instructions, memory, retrieved evidence, tools, permissions, summaries, and live environment data.
- [Silent Failure in AI Systems](https://hermes-labs.ai/topics/silent-ai-failure): Silent AI failures return plausible outputs while hiding missing evidence, lost context, false completion, retrieval gaps, or unsupported confidence. How to detect and prevent them.
- [Prompt Linters Do Different Jobs](https://hermes-labs.ai/topics/prompt-linters): What a prompt linter is, how linting differs from prompt testing, guardrails, and monitoring, and how to choose the right check for each failure.

Index: https://hermes-labs.ai/topics

Key findings: 1,461 controlled experiments, conducted primarily on GPT-4o, underpin the taxonomy paper (DOI 10.5281/zenodo.19042468). 105 published external pull requests merged in 2026, including 52 code, workflow, typing or maintenance contributions across open-source infrastructure. Current totals and dated record: https://hermes-labs.ai/open-source/contributions, with the complete merged AI/framework-fix list in the canonical ledger of merged external pull requests: https://hermes-labs.ai/contributions.json.

## Open-source AI agent reliability tools

Tools for the AI failures that surface after the demo. The public catalog records the current company-owned tools and their workflow roles. Public source makes a tool inspectable; it is not certification or a production-fitness guarantee.

- Machine-readable catalog of all listed projects (canonical identifiers, task descriptions, install surfaces, maturity boundaries), generated from the same record as the catalog page: https://hermes-labs.ai/software-catalog.json. Documented relations between projects (complements, differs-from; none depend on another): https://hermes-labs.ai/capability-graph.json
- Name resolver for every project, package, display and legacy name (for example cogito-ergo → fidelis, pygate-ci → quick-gate-python), each resolving to one canonical id: https://hermes-labs.ai/aliases.json
- LintLang product home: https://lintlang.ai/ is the LintLang product URL. https://hermes-labs.ai/lintlang is the Hermes Labs page about the product, and https://github.com/hermes-labs-ai/lintlang is the source repository and canonical README.
- LintLang product page on this site: https://hermes-labs.ai/lintlang
- LintLang 简体中文快速上手 (Simplified Chinese quickstart, a translation: the English page and README are authoritative, commands are identical and carry no version number, language zh-Hans): https://hermes-labs.ai/zh-cn/lintlang
- LintLang canonical documentation: https://lintlang.ai/docs
- Little Canary canonical documentation: https://littlecanary.ai/docs
- LintLang setup for GitHub Actions, pre-commit, Claude Code, Gemini CLI, MegaLinter, and GitHub Code Scanning: https://hermes-labs.ai/lintlang#integrations (text copy with per-surface version pins: https://hermes-labs.ai/lintlang.md)
- LintLang FAQ and comparison: https://hermes-labs.ai/lintlang/faq answers why use LintLang and how it compares with Agent Lint, promptfoo, Giskard, and NVIDIA NeMo Guardrails by jobs and boundaries, each row from the other project's official documentation, with a machine-readable copy at https://hermes-labs.ai/lintlang/faq.json.
- Langstate product page: https://hermes-labs.ai/langstate
- zer0dex 0.1.2 local-memory product page and source: https://hermes-labs.ai/zer0dex and https://github.com/hermes-labs-ai/zer0dex
- zer0dex setup, local Ollama prerequisites, loopback API, configuration, and limits: https://hermes-labs.ai/zer0dex/docs
- Fidelis Memory 0.2.0 product page and source: https://hermes-labs.ai/fidelis and https://github.com/hermes-labs-ai/fidelis
- Fidelis Memory installation, local note-to-retrieval verification, MCP client setup, troubleshooting, and limits: https://hermes-labs.ai/fidelis/docs
- Native Pi prompt-time recall from current Fidelis source (separate from MCP tools and the memory store): https://hermes-labs.ai/fidelis/docs#pi-prompt-time-recall
- Hermes Blind product and documentation: https://hermes-labs.ai/hermes-blind and https://hermes-labs.ai/hermes-blind/docs; catalog entry: https://hermes-labs.ai/open-source#hermes-blind. The `hermes-blind 0.3.2` CLI installs with `pip install hermes-blind==0.3.2`. It extracts a local Claude Code or Codex session anchor; each host skill still needs a Python runner, and recovery output can contain user-authored text.
- Hermeneutic documentation: https://hermes-labs.ai/hermeneutic/docs. Its local fixed-rule draft gate reports PASS, RISK, or a distinct input error; it does not establish a claim's truth or use correction retrieval in that check.
- Hermes Rubric documentation: https://hermes-labs.ai/hermes-rubric/docs. It requires a configured model backend and reports citations, coverage, hedge fields, and a receipt; its aggregate is advisory.
- Gemini CLI extension installation for Fidelis Memory (from its v0.2.0 release; needs uv): https://hermes-labs.ai/open-source#fidelis; exact commands in the "Gemini CLI extension installation" section of https://hermes-labs.ai/llms-full.txt
- Static checks and drift gates: [lintlang 0.8.2](https://github.com/hermes-labs-ai/lintlang) includes deterministic H1.6 pairwise differentia analysis for tool descriptions within one parsed input and SARIF 2.1.0 output for GitHub Code Scanning. Try it once with `uvx --from lintlang==0.8.2 lintlang scan AGENTS.md` or install the release with `pip install lintlang==0.8.2`. Neither its static findings nor the Code Scanning integration establish runtime tool-selection improvement or runtime safety. [hermeneutic 0.1.12](https://github.com/hermes-labs-ai/hermeneutic) mines corrections from chat logs (prior response, user correction, repair) and runs a preflight drift gate (`pip install hermeneutic==0.1.12`). Neither tool proves the resulting system or response is correct.
- LintLang public ecosystem evidence: Character.AI's public Larch repository pins lintlang 0.3.1 as a blocking step in recurring CI. The linked public job records a successful execution, and Larch's linting reference links hermes-labs-ai/lintlang as the upstream. Sources: [Larch CI workflow](https://github.com/character-ai/larch/blob/ef7ee4b7f946f29fa51981f5422a1a93e83c79a7/.github/workflows/ci.yaml), [Historical Larch requirements pin](https://github.com/character-ai/larch/blob/ef7ee4b7f946f29fa51981f5422a1a93e83c79a7/.github/workflows/requirements-agent-linters.txt), [Successful public execution](https://github.com/character-ai/larch/actions/runs/32180477808/job/95852067633), [Larch upstream linting reference](https://github.com/character-ai/larch/blob/210d08a8f6c1b0dd14c27b709c66471bd31a5636/docs/linting.md); An independent maintainer provides Gentoo packaging in the unofficial Haven overlay, which tracks upstream releases across successive ebuild bumps. Its metadata.xml records hermes-labs-ai/lintlang as the upstream remote. Sources: [Haven overlay package](https://github.com/thehaven/haven-overlay/tree/fbed4463d4e213a3bce98babbfc72ef78722d255/dev-util/lintlang), [Upstream package metadata](https://github.com/thehaven/haven-overlay/blob/fbed4463d4e213a3bce98babbfc72ef78722d255/dev-util/lintlang/metadata.xml); The Agent Lint maintainer attributes parts of that product's roadmap to useful ideas from lintlang. Sources: [Agent Lint maintainer attribution](https://github.com/zhupanov/agent-lint/issues/192); MegaLinter's third-party plugin catalog [lists lintlang as the AI_LINTLANG linter](https://github.com/oxsecurity/megalinter/blob/9d4e685d4f31cecdd53743a08704cd23d9c01811/docs/plugins.md) and resolves the descriptor from this repository, a record that documents an opt-in plugin entry, not a bundled MegaLinter default.
- Observe and guard: [little-canary 0.5.0](https://github.com/hermes-labs-ai/little-canary) is an experimental prompt-injection preflight (`pip install little-canary==0.5.0`); efficacy and false-positive behavior depend on the models and evaluation distribution.
- Evaluate: [hermes-rubric 1.2.3](https://github.com/hermes-labs-ai/hermes-rubric) binds advisory scores to cited evidence (`pip install hermes-rubric==1.2.3`); its published agreement result still needs independent reproduction. [hermes-jailbench 0.2.2](https://github.com/hermes-labs-ai/hermes-jailbench) runs a repeatable battery of known-pattern jailbreak attacks against an Anthropic or OpenAI-compatible model endpoint and classifies each response as refusal, partial, or compliance with deterministic keyword heuristics, including a dry-run mode that needs no API key (`pip install hermes-jailbench==0.2.2`); it covers known patterns scored by keywords, so a clean run is not evidence of safety against novel attacks.
- Memory and recovery: [Langstate 0.2.4](https://github.com/hermes-labs-ai/langstate) compresses older conversation turns into a visible scaffold, preserves recent turns, and emits receipts for named literal facts; run `langstate demo`. It is a lossy prototype, not a semantic-fidelity guarantee. [Fidelis Memory 0.2.0](https://pypi.org/project/fidelis-memory/) returns your original stored passages verbatim, with no LLM call in the default retrieval path (`pip install fidelis-memory==0.2.0`); its repository, import name, and CLI remain `fidelis`, while the separate PyPI package named `fidelis` is unrelated. The unaffiliated ai-memory-comparison project includes a [source-referenced feature comparison of fidelis 0.0.9](https://github.com/carsteneu/ai-memory-comparison/blob/2b5abafb6dff00cb514ce18b0c64bb567ea95cf4/evidence/fidelis.md), dated 28 May 2026. [hermes-blind 0.3.2](https://github.com/hermes-labs-ai/hermes-blind) extracts turn-one goals from Claude Code or Codex session logs into a compact recovery anchor (`pip install hermes-blind==0.3.2`); it does not detect drift or prove improved model behavior. [zer0dex 0.1.2](https://github.com/hermes-labs-ai/zer0dex) is a compact developer-preview reference implementation for dual-layer local memory (`pip install zer0dex==0.1.2`); benchmark implications are not independently reproduced.

### Choose a tool by job

One line per listed project: the job it does, and the boundary to read before selecting it. The same catalog is machine-readable at https://hermes-labs.ai/software-catalog.json.

- Lint agent instruction files (AGENTS.md, CLAUDE.md, SKILL.md), system prompts and tool schemas for overlapping or vague tool descriptions, mixed output formats, schema gaps and missing stop conditions: lintlang. It does not detect semantic contradictions between two instructions. Static and deterministic; a clean scan does not prove runtime safety.
- Recover the original goal of a long Claude Code or Codex session from its first user turn: hermes-blind. Does not detect drift.
- Mine prior corrections from supported chat logs and run a preflight drift check on a new draft: hermeneutic. Advisory; not proof a draft is correct.
- Compress older conversation turns into visible state and check that named literal facts survived: langstate. Lossy prototype.
- Store agent memory locally and return the original passages verbatim, with no LLM call in the default retrieval path: fidelis (PyPI package fidelis-memory). Independent install-to-recall proof remains open.
- Study a compact dual-layer memory pattern, a human-readable markdown index paired with semantic vector retrieval: zer0dex. Developer-preview reference implementation; its benchmark implications are not independently reproduced.
- Probe untrusted input with a sacrificial canary model before the primary model acts: little-canary. Experimental prompt-injection preflight.
- Regression-test a model endpoint against known-pattern jailbreak attacks, scoring refusal, partial or compliance: hermes-jailbench. Known patterns only.
- Score an AI artifact with a rubric whose scores are bound to cited evidence, with a reproducibility receipt: hermes-rubric. Scoring is a model call.

Catalog with per-tool evidence boundaries: https://hermes-labs.ai/open-source
Research context: https://hermes-labs.ai/research

## Upstream contributions

105 published external pull requests merged in 2026, including 52 code, workflow, typing or maintenance contributions across open-source infrastructure. The contribution record (https://hermes-labs.ai/open-source/contributions) includes LangChain, Microsoft Semantic Kernel and DSPy, plus PyTorch Ignite, Optuna, React Router, Nuxt, Cloudflare Workers, MobX, ngrx, and more. Release evidence for LangChain #35544 and Semantic Kernel #13610: https://github.com/langchain-ai/langchain/releases/tag/langchain-anthropic%3D%3D1.3.5 and https://github.com/microsoft/semantic-kernel/releases/tag/python-1.41.1

DSPy #9978 and Semantic Kernel #13635 each have a worked case study linked from the canonical ledger.

Detail with pull-request links: https://hermes-labs.ai/open-source/contributions

## External outcomes

- Public data integrity: Roli Bosch reported that the CISA ADP vector for CVE-2026-14216 calculated to 6.5 while its stored base score said 5.3. CISA acknowledged the submission, republished the corrected record, and the official CVE corpus records the 5.3 to 6.5 correction. The vector and Medium severity category did not change. This accepted public-data correction brings the stored score into agreement with its vector. Evidence: https://github.com/cisagov/vulnrichment/issues/333, https://github.com/cisagov/vulnrichment/commit/4dad1386622ef620991b1f7e886a92a9fd9c7d73, and https://github.com/CVEProject/cvelistV5/commit/fa52a90c974f64966065f0beb9f4ffcefaf1208f


## Case studies

Worked examples of reliability engineering across AI frameworks, tools, and public data, with the evidence, attribution, and external outcome stated per case.

- [Hermes Labs CISA CVSS Consistency Correction](https://hermes-labs.ai/case-studies/cisa-vulnrichment-score-consistency): a bounded audit found one CVSS score/vector mismatch; the system corrected an unsupported history inference and stopped at the external-action boundary, Roli Bosch approved the exact filing, and CISA acknowledged and corrected the record. Selected machine-readable trace: https://hermes-labs.ai/case-studies/cisa-vulnrichment-score-consistency/trace.json.
- [Making LangChain’s agent tool binding compatible with Claude’s extended thinking](https://hermes-labs.ai/case-studies/fixing-langchain-thinking-tools-crash): a 20-line guard in ChatAnthropic.bind_tools() drops a forced tool_choice when Claude extended thinking is enabled (the Anthropic API rejects that combination with a request-time 400), warns that tool calls are not guaranteed, and keeps auto and thinking-off bindings; two test functions cover eight scenarios; root cause credited to the community reporter; merged upstream (LangChain PR #35544).
- [When the System Prompt Vanishes: A Silent Failure in Semantic Kernel](https://hermes-labs.ai/case-studies/fixing-semantic-kernel-deleted-system-prompts): truncating a long conversation silently deleted the system prompt, so the model quietly stopped following its instructions; the Python instance was reproduced, the fix ported, and merged (Semantic Kernel PR #13610).
- [Auditing Agent Memory: A Silent Retrieval Failure in mem0 v2.0](https://hermes-labs.ai/case-studies/auditing-mem0-retrieval-scoring): a cited Chroma reproduction of an all-1.0 score collapse (mem0 #4999) and a separately tested Redis patch (PR #5250, unmerged) whose conversion a maintainer acknowledged as included in the merged eleven-backend sweep (PR #5391); earlier community diagnosis is credited.
- [Forensic Diagnosis of a Closed-Source Hang: Claude Code --resume](https://hermes-labs.ai/case-studies/forensic-diagnosis-claude-code-resume-hang): a falsification-first diagnosis of a closed-source CLI hang with no source access; five cause-classes tested by direct measurement (two falsified, two partially falsified, one surviving unconfirmed) and a falsification log with three confirmation tests filed upstream (claude-code #55241, auto-closed stale with no linked fix).

Full set: https://hermes-labs.ai/case-studies

## Technical reports

Investigations with a reusable lesson, published with the upstream status stated as it stands. A technical report is not a case study and does not claim adoption, deployment or endorsement.

- [When a Failed LangGraph Router Disappears on Resume](https://hermes-labs.ai/reports/langgraph-failed-router-disappears-on-resume): HL-TR-2026-001. A conditional router that fails on the first LangGraph 1.2.11 invocation is skipped on resume, which returns normally without running the downstream node; minimized fixture with an ordinary-node control, pinned source mechanism, two independent reproductions and a credited application-level containment. Upstream issue #8834 remains open; no framework fix or containing release is claimed.
- [Evidence Before Remediation: Why a Permission Error Does Not Identify Its Cause](https://hermes-labs.ai/reports/nono-evidence-before-remediation): HL-TR-2026-003. Nono #1816 gates path and network diagnostic guidance on observed evidence and Nono Packs #29 requires Nono provenance before recommending a permission change; both merged 9 September 2026, and core v0.76.0 contains #1816 by commit ancestry. No Packs release, binary validation or deployment is claimed.

Index: https://hermes-labs.ai/case-studies#technical-reports

## Writing

Long-form essays on AI failure modes, agent reliability, runtime controls, and the language layer of production AI. Full text at /archive; each post links back to the original on Substack.

- [Writing index](https://hermes-labs.ai/archive): all essays.
- [Why your AI lies when the data is right](https://hermes-labs.ai/archive/why-your-ai-lies-when-the-data-is): on silent failure modes, null-result omission, and the evidence layer teams are not building.
- [Your AI isn't forgetting its instructions](https://hermes-labs.ai/archive/your-ai-isnt-forgetting-its-instructions): on silent instruction relaxation and verifying behavior by its effect on static code.

## Contact

- Email: roli@hermes-labs.ai
- GitHub: https://github.com/hermes-labs-ai
- LinkedIn: https://www.linkedin.com/company/hermeslabs/
- Substack: https://hermeslabs.substack.com/
- X: https://x.com/hermeslabsai