AI auditing library

A voluntary framework for organizing AI risks, responsibilities, measurement, and management. Use it to frame the questions your audit needs to answer.

A framework for evaluating models and agents, with reusable tasks, tools, scorers, and evaluation logs. Start here to design repeatable tests.

A benchmark environment for testing prompt-injection attacks and defenses on tool-using agents. Check its tasks and threat model before comparing results.

A maintained reading list of papers, tools, benchmarks, and standards covering agent reliability, monitoring, failure attribution, and accountability.

A benchmark for auditing agent failures. The offline PRE quickstart runs without model API keys; other benchmark paths have additional setup requirements.

An ecosystem page connecting tools for inspecting, enforcing policies on, and reviewing agent actions. Follow the component repositories for current capabilities.

Broad survey framing LLMs as the basis for autonomous agents, covering single-agent, multi-agent, and human-agent settings.

Systematic review of LLM-based autonomous agents with a unified construction framework and a section on agent evaluation strategies.

Survey of LLM multi-agent systems organized by agent profiling, communication, and capability growth, with open challenges.

Position-style survey of unresolved problems in multi-agent systems, including task allocation, reasoning debate, context, and memory.

Survey of agent evaluation and benchmarks, highlighting gaps in cost, safety, and robustness.

TrustAgent taxonomy separating intrinsic components (brain, memory, tool) from extrinsic ones (user, other agents, environment), mapping attacks, defenses, and evaluation onto each.

Accountability-focused position paper proposing agent identifiers, real-time monitoring, and activity logging as measures to make agent activity answerable.

Taxonomy of the artifacts and trace data to record across an agent's lifecycle, drawn from a mapping study of observability tools.

Defines automated failure attribution for LLM multi-agent systems and releases the Who&When dataset of annotated failure logs labeling the responsible agent and the decisive error step.

Derives MAST, a 14-mode failure taxonomy in three groups (system design, inter-agent misalignment, task verification), from 150 expert-examined traces, and releases MAST-Data, 1,642 annotated traces across seven multi-agent frameworks.

Annotates 148 single- and multi-agent traces with 841 errors under a reasoning, planning, and execution taxonomy, and shows long-context models score near 11% at locating them.

Curates the TracerTraj attribution dataset and trains AgenTracer-8B to identify the responsible agent and step in long failed trajectories.

Proposes ECHO, an agent- and step-level attribution method combining hierarchical context, objective analysis, and consensus voting for subtle reasoning errors.

Generates controlled faults to build a 9,533-trajectory dataset with annotated faulty agents and error modes for training and evaluating attribution methods.

Synthesizes constraints, checks them step by step, and records the results in an auditable validation log that localizes the critical failure step within a 10-category taxonomy across three domains.

Introduces TraceElephant, a failure-attribution benchmark with fully observable traces and reproducible environments, reporting attribution accuracy up to 76% higher with full execution traces than with output-only logs.

Releases AgentFail, a collection of 307 failures from low-code agent platforms, and characterizes them by manifestation, root cause, and repair strategy at the workflow-node level.

Frames attribution as dependency-guided coarse-to-fine search that separates root-cause steps from inherited downstream errors, evaluated on the Who&When benchmark.

Builds Who&When Pro, 12,326 failed trajectories with golden labels across three modalities and 26 source benchmarks, constructed by replaying a successful prefix exactly and then injecting a single failure.

Introduces AgentErrorTaxonomy across memory, reflection, planning, action, and system operations, releases AgentErrorBench of annotated failure trajectories from ALFWorld, GAIA, and WebShop, and reports 24% higher all-correct accuracy for its AgentDebug root-cause debugger.

Uses a language model only to encode execution logs into temporal semantic sequences and scores steps with parameter-efficient temporal and attention modules, cutting inference time by 79% against the fastest language-model-based method on Who&When.

Prunes trajectory context by symptom-driven backward tracing and gates memory updates on executable evidence, so ErrorProbe needs no annotation and transfers without retraining.

Releases MP-Bench and argues that reported language-model weakness at attribution largely reflects single-root-cause benchmark design rather than model capability.

Trains one-class neural controlled differential equations on 100 successful trajectories alone and scores deviation, gaining 20% in-domain F1 at 200 to 5000 times the speed of prompting baselines.

Applies filtration-based conformal prediction over trajectories, returning contiguous step sets with finite-sample distribution-free coverage guarantees.

Releases SearchAuditBench, 1,243 failed deep-search trajectories averaging 73.1 messages and 65.1K tokens, on which the strongest baseline reaches a 26.6% end-to-end pass rate.

Annotates 2,790 deep-research trajectories and filters them into TELBench, a 1,000-instance benchmark for erroneous-span and first-error localization, on which DRIFT gains up to 30 percentage points.

Runs a tool-augmented diagnostic loop over short-term memory instead of loading a whole trajectory, gaining 20% on Who&When at a 1M token budget and 19% on the TRAIL GAIA subset at a 25K token budget.

Annotates 1,794 CLI coding trajectories over 63,000 steps from 7 models and 3 scaffolds on Terminal-Bench, finding that failures are mostly epistemic, start early, and stay hidden until recovery is impossible.

Localizes error steps in agentic coding trajectories and releases RootSE, a benchmark of 102 failure trajectories with human-annotated error steps spanning localization, patch generation, verification, and environment failures.

Mines 13,602 issues and pull requests from 40 agentic repositories, samples 385 faults, derives 34 fault types, and validates the taxonomy with 145 practitioners.

Releases CUAErrorBench, 204 human-annotated OSWorld failures, and raises joint subtype-and-step diagnosis from 11.2% to 19.6% on the Claude-agent split.

Models a run as one graph with two edge layers, execution for what ran in what order and dependency for what each step relied on; across six corpora the dependency layer predicts failure where run size carries little signal, and the execution layer localizes the faulting step.

Measures tool-using agent reliability along three axes: consistency under repeated runs (pass^k), robustness to semantically equivalent task perturbations, and fault tolerance under injected tool and API failures.

Benchmarks agents in simulated tool-agent-user dialogues and introduces the pass^k metric, which scores whether an agent solves the same task on all of k independent trials.

Documents variation in output text and task accuracy across repeated runs under nominally deterministic settings (temperature 0 and fixed decoding), identifying one source of run-to-run inconsistency in agent pipelines.

Runs the same agent on the same task many times, finding several distinct ReAct-style action paths per ten runs, with inconsistent-path tasks scoring lower than consistent-path ones.

Quantifies how tool-using agents degrade under user-side instruction noise and tool-execution noise, reporting accuracy drops across models under realistic perturbations.

Trains agents to detect and recover from tool malfunctions (timeouts, API exceptions, inconsistent outputs) that otherwise trigger cascading reasoning errors and task abandonment.

Wraps tool use in progress-aware transactions that commit only after per-resource frontiers rule out earlier conflicting work, at microsecond-scale wrapper overhead.

Names semantic rollback attacks, in which an agent re-synthesizes a slightly different request after restore and the server treats it as new, enabling duplicate payments and credential reuse.

Compares A2A, ACP, ANP, and Agora on success, latency, message overhead, and failure robustness, finding that completion time in the Streaming Queue scenario varies by up to 36.5% across protocols and mean end-to-end latency differs by 3.48 seconds.

Argues that a single success rate hides operational flaws and proposes twelve metrics decomposing agent reliability into consistency, robustness, predictability, and safety.

Proposes reliability metrics for long-horizon tasks (reliability decay curve, variance amplification, graceful degradation, meltdown onset) and evaluates 10 models over 23,392 episodes.

Sets how many identical repeated runs a measurement needs before it is stable, using generalizability-theory reliability, simulation-based power, block-bootstrap convergence, and drift tests, with external validation on three repeated-query corpora collected by other groups including llm-stability.

Decomposes run-to-run variance in LLM brand responses across resampling, paraphrase, model, and query language over 12,933 responses, then converts the components into a query-budget allocation; reported components are point estimates with confidence intervals pending.

Evaluates monitoring protocols intended to improve deployment safety when a model tries to subvert oversight, using a trusted weaker model and limited trusted labor.

Screens user prompts and model responses against a safety risk taxonomy as an input-output filter for agent applications.

Reads safety requirements, generates a check plan, and compiles the plan into guardrail code that checks a target agent's runtime actions without retraining.

Captures per-agent inputs and outputs to predict task outcomes ahead of time and apply real-time corrections when a malicious agent threatens the multi-agent run.

Runs a graph neural network over the multi-agent utterance graph to flag compromised agents and then applies topological intervention to remediate the attack mid-run.

Generates and optimizes safety checks during execution, adapting them to task-specific and systemic risks over an agent's lifetime rather than relying on a fixed rule set.

Models a multi-agent collaboration as a temporal attributed graph and uses an unsupervised encoder-decoder to detect anomalous nodes and edges as errors propagate.

Watches the live execution graph, scores nodes, edges, and paths against security policies, and intervenes on single-point faults or collusion.

Estimates the likelihood of reaching an unsafe future state with a discrete-time Markov chain so the monitor can warn and intervene before a violation occurs.

Intercepts sensitive operations in a computer-use agent and halts execution until a security audit clears, correlating task context with system traces in real time.

Uses a Siamese recurrent autoencoder to classify a whole agent trajectory as either a wrong plan for the task or a malformed plan structure, reporting 32 ms per sample and a 17 to 27 times speedup over language-model judge baselines.

Shows that an untrusted model which knows the control protocol and the monitor can embed prompt injections in its own outputs to evade diverse language-model monitors on two AI control benchmarks, and that the Defer-to-Resample protocol amplifies the injection into a best-of-n attack.

Stress-tests monitors watching a stronger agent for covert misbehavior across SHADE-Arena and CUA-SHADE-Arena, showing that monitor scaffolding matters more than monitor awareness and that escalating only pre-flagged cases to human reviewers raises the true-positive rate by about 15 percent at a false-positive rate of 0.01.

Measures how language-model monitors of coding agents degrade as transcripts grow, reporting that Opus 4.6, GPT 5.4, and Gemini 3.1 miss dangerous actions 2x to 30x more often after 800K tokens of benign activity than when the same actions appear alone.

Enforces agent-declared policy in the kernel with eBPF and an information-flow language, covering indirect execution paths that tool-call interception cannot observe, at 1.9% to 8.4% overhead.

Assembles 40 covert attacks across 11 evasion categories, on which an Opus 4.6 monitor with extended thinking catches 32% overall and never catches 20 of the 40 at a 1% false-positive rate over ten trials.

Defines a trigger, predicate, and enforcement-action rule language that prevents over 90% of unsafe code-agent executions and all hazardous embodied actions at millisecond overhead.

Induces typed-step adapters offline and then trains prefix-risk scorers for online warnings, reaching 0.900 AUPRC on WebArena and gaining 0.137 on average over raw-text controls across four benchmarks.

Trains four open-weight guard models from 0.8B to 8B parameters on about 1,000 taxonomy-guided samples and deploys them as a training-free online guardrail.

Trains a 4B guard model on automatically generated safe and unsafe step-level trajectories to check tool actions before execution and audit completed trajectories, reducing mean attack success rate by 77.3% relative to no guard across AgentDojo and AgentDyn.

Reconstructs traces as control-flow, data-flow, and program-dependence graphs and type-checks policy over them, cutting AgentDojo attack success to 3 percent at 1 percent utility cost.

Replays each proposed tool call under control-attenuated counterfactual views and blocks calls that vanish without the untrusted content, reaching 0% attack success across four AgentDojo suites at about 3% utility cost and holding adaptive attacks to single-digit success rates.

Pairs 263 act and abstain tasks across 42 executable environments, on which the best of 17 frontier models reaches 59.5% paired accuracy and some agents act irreversibly before recognizing a reason to stop.

Compiles temporal safety properties to first-order logic and enforces them at generation time with an SMT solver, raising conformance to 100% for two frontier models while also raising utility.

Combines a deterministic rule-based trajectory monitor with an advisor model consulted only on drift, gaining up to 15.2% resolution rate on SWE-bench Verified and Pro at $0.08 per instance.

Hides human-validated blockers in software-engineering and text-to-SQL tasks and scores selective escalation with an Ask-F1 metric.

an open-source toolkit that adds programmable rails to LLM applications and agents, intercepting requests to enforce Colang-defined policies and validate tool inputs and outputs before and after a call. Apache-2.0, 2023-present.

an open-source Python framework that runs input and output guards in an application, composing validators from Guardrails Hub to detect and mitigate risks such as toxicity, PII exposure, and hallucination. Apache-2.0, 2023-present.

a layered guardrail framework combining PromptGuard 2 for jailbreak and injection detection, AlignmentCheck as an experimental chain-of-thought auditor that inspects agent reasoning for goal hijacking, and CodeShield static analysis across 8 languages for insecure generated code. Llama 3.2 Community License, which restricts redistribution and is not an OSI-approved open-source license, 2023-present.

Uses data-flow lineage to expose decision pipelines, including inputs, decisions, and downstream effects, as a basis for accountability and audit.

Proposes lifecycle audit trails that record technical lineage and governance decisions (approvals, waivers, attestations) in tamper-evident logs for LLMs in high-stakes use.

Argues that responsibility becomes assignable only when agents emit traceable, attributable records across the lifecycle, and formalizes a causal attribution function and responsibility tensor.

Defines five auditability dimensions for tool-using agents (action recoverability, lifecycle coverage, policy checkability, responsibility attribution, evidence integrity) and an Auditability Card.

Proposes Agent-BOM, a unified hierarchical attributed graph over agent execution that captures capability bindings, cognitive-state evolution, memory contamination, and cross-agent risk propagation as one structured record for post-hoc security auditing.

Treats an append-only event log as the canonical record and derives the working graph from that log, supporting replay, forks from prior events, and end-to-end lineage for each run.

Builds lineage records from agent execution traces and abstracts them into behavioral rules; ships TraceAegis-Bench with labeled benign and abnormal traces.

Records execution lineage from prior legitimate runs and uses it to bound and account for an agent's later actions at runtime.

Surveys evidence tracing and execution lineage for process-level accountability, organizing trace sources, evidence units, and lineage relations for auditable agent systems.

Signs each memory entry per principal over an RFC 6962 Merkle log and refuses sensitive actions whose active justification descends from external content, driving attack success to zero on three memory-poisoning workloads and six AgentDojo banking pairs at sub-millisecond overhead.

Leaves action choice distortion-free on a selection channel and keys a tally channel on log structure alone, so a reseller who can rewrite the log cannot erase attribution; detection survives deletion of 70% of steps.

self-hosted governance for AI agents with role-based execution, human approval gates, and a hash-chained, Ed25519-signed audit log of every action. AGPL-3.0, 2026-present.

runtime policy enforcement for AI agents with a cryptographic audit trail, human-in-the-loop approvals, and a kill switch, implemented as a wrapper around agent code. MIT-licensed, 2026-present.

commit-time audit engine for AI coding agents that checks git diffs and any available task log against 24 rules, 17 of them enabled by default, recording each result in a local hash-chained history with optional HMAC signing. A doctor command checks chain integrity over the latest 500 history entries, and an MCP server aggregates the local trails by agent. MIT-licensed, 2026-present.

dependency-free recorder that seals tool calls, model calls, and approvals into a SHA-256 hash-chained JSONL log verifiable without trusting the operator, with optional RFC 3161 timestamping and OpenTelemetry GenAI span ingestion. Apache-2.0, 2026-present.

policy enforcement, agent identity, and execution sandboxing with a Merkle-chained audit trail and a Decision BOM, published with control mappings to the OWASP Agentic Top 10, NIST AI RMF 1.0, the EU AI Act, and SOC 2. MIT-licensed, 2026-present; public preview with breaking changes expected.

open specification and Python SDK for hardware-attested agent trust records that bind model, policy, data class, tool calls, and TEE attestation into one signed artifact a third party can verify offline, with a conformance test suite. CC BY 4.0 spec, Apache-2.0 tooling, 2026-present.

MCP-native observability and audit-trail platform that records LLM calls, tool invocations, and decisions in an append-only, SHA-256 hash-chained, verifiable event log. MIT-licensed, 2026-present.

records which inputs a consequential agent action relied on, re-evaluates that action against current state, and reverses the committed action through a pluggable rail when its supporting conditions no longer hold. Apache-2.0, 2026-present.

verification-first agent that turns task contracts into deterministic SHA-256 proof packets, rejects missing, stale, or contradictory evidence, and gates external actions behind human approval. MIT-licensed, 2026-present.

bounded agent loop with a caller-injected host that emits canonical Trace and Evidence documents for a run and checks them by internal replay before returning them. It is reached through the project's Rust API, so it does not attach to an agent written in another language, and the surrounding language project is pre-alpha. Apache-2.0, 2026-present.

self-hostable database and workspace for AI agents where scoped credentials can require data, document, schema, skill, and app mutations to arrive as ChangeRequests with exact diffs, submitter attribution, comments, review decisions, commits, and history before approved changes become canonical. MIT-licensed, 2026-present.

Static security analysis for LLM agent code and configuration, with tool-boundary taint tracking and MCP config auditing; the tool itself is listed under Scanners.

A benchmark of 1,054 cases that measures how often tool-integrated LLM agents follow malicious instructions embedded in tool outputs.

A systematization of security risks and attack vectors in agentic systems that combine language models with tools, retrieval, and autonomous decision loops.

Extracts the control and data flow from the trusted query so untrusted retrieved data cannot affect the program flow, and enforces capability-based policies when tools are called, solving 77% of AgentDojo tasks with provable security against 84% for an undefended system.

Presents six design patterns that constrain a tool-using agent once it ingests untrusted input, with a utility and security trade-off analysis for each.

Formalizes 10 prompt-injection attacks, memory poisoning, a Plan-of-Thought backdoor, 4 mixed attacks, and 11 defenses across 13 backbones, 10 scenarios, and over 400 tools, reaching a peak average attack success rate of 84.30 percent.

Shows that MINJA injects malicious records through ordinary queries and observations alone, with injection success above 90% in most configurations across three agents.

Follows 310 memory-poisoning cases from 48 contexts through write, execute, and forget stages across 24 harness, backend, and model combinations, reporting 84.2% persistence and 50.3% full-chain success, with selective repair at 56.1% among successfully poisoned cases.

Decomposes an adversarial goal into innocuous sub-steps along the navigation path, raising attack success on 3 of 6 computer-use agents by up to 31.2 points across 480 examples.

Introduces StakeBench, which scores prompt-injection harm by the affected stakeholder (user, seller, platform) over 264 executable cases from 22 templates on a live e-commerce environment, where indirect injection succeeds 41.67% to 68.16% of the time and the same agent shows a distinct failure profile per stakeholder.

Threat model for Model Context Protocol clients showing how unvalidated tool metadata enables prompt injection and tool poisoning, with static-analysis and behavioral defenses.

A survey of security and safety issues across the Model Context Protocol ecosystem, including tool poisoning, supply-chain risk, and proposed mitigations.

Builds MCPZoo from 64,611 unique MCP servers, over 37,288 of which support dynamic analysis, and finds existing scanners flag 96.89% of servers as risky at 45.53% average alert precision.

Finds with structure-aware static analysis that 9.93% of 19,200 description-code pairs from 2,214 real servers misstate what the tool code does.

Measures 7,973 live remote servers, of which 40.55% expose tools with no authentication, and reports 325 flaws and 9 CVE identifiers from 119 OAuth-enabled servers.

Audits skill-package reuse across expression, implementation, and operational traces, reaching AUROC 0.938 and F1 0.898 on 820 transformed positives against 751 negative controls, then audits 36,446 marketplace skills.

Shows self-extracting skill packing bypasses all 8 tested scanners at over 90% across 1,613 in-the-wild malicious skills, while its sandbox auditor detects 97% of benchmark attacks at a 2% false-positive rate and 87% of real-world malicious skills.

Collects 263 marketplace skills across 7 threat categories, on which the safest of 3 frameworks and 13 models still executes unsafe actions about 17% of the time.

static security scanner for LLM agents that flags prompt-injection sinks, audits MCP configuration, and runs tool-boundary taint analysis. Its rules map to the OWASP Agentic Top 10 (2026), and it supports LangChain, CrewAI, and AutoGen. MIT-licensed, 2026-present.

an LLM vulnerability scanner that probes a model or any REST-accessible endpoint for prompt injection, jailbreaks, data leakage, and toxic generation, with detectors that score each probe. Apache-2.0, 2023-present.

open-source scanner that maps agentic workflows through static analysis, detects MCP servers and tools, and reports OWASP-aligned vulnerabilities. It supports LangGraph, CrewAI, OpenAI Agents, AutoGen, and n8n. Apache-2.0, 2025 (last commit 2025-11-27).

security scanner that auto-discovers the MCP servers, agent tools, and skills installed on a machine across 13 agent harnesses including Claude Code, Cursor, VS Code, Windsurf, Gemini CLI, and Codex. It flags more than 15 risk classes including prompt injection, tool poisoning, tool shadowing, toxic flows, malware payloads, and hardcoded secrets. Apache-2.0, 2025-present; it requires a Snyk API token, so it is an open-source client to a commercial service.

An evaluation framework that generates synthetic multi-agent interaction datasets and standardizes the comparison of graph-based anomaly detectors over agent communication graphs.

2,294 software-engineering tasks drawn from real GitHub issues and their fix pull requests across 12 Python repositories, with unit-test verification of each generated patch.

An open-source coding-agent framework with released SWE-bench trajectories that contain step-by-step tool calls and edits for trace-level study.

A suite of eight interactive environments for measuring multi-turn decision-making of LLM agents, with logged interaction records across many models.

466 real-world questions that require multi-step reasoning, web browsing, and tool use, with a single correct answer per item for unambiguous scoring of assistant behavior.

A reproducible environment of self-hosted websites with long-horizon tasks scored by functional correctness of the end state, supporting repeatable agent evaluation.

369 executable computer-use tasks on Ubuntu, each with a setup configuration and an execution-based checker, in an environment that also supports Windows and macOS, on which humans complete over 72.36% of tasks while the best evaluated model reaches 12.24%.

Terminal-Bench 2.0 curates 89 command-line tasks, each with its own container environment, a human-written solution, and verification tests, on which frontier models and agents score below 65%.

750 interactive coding tasks over 9 simulated apps exposing 457 APIs, on which GPT-4o solves about 49% of normal tasks and 30% of challenge tasks.

ToolEmu emulates tool execution so agents can be stress-tested on 144 cases without real side effects; the safest agent still fails 23.9% of the time and 68.8% of flagged failures are judged valid.

Extends τ-bench to dual control where the user also acts on the shared environment, modeling Telecom as a Dec-POMDP and separating agent reasoning errors from coordination errors.

Nine partially observable environments scored by a fine-grained progress rate that credits incremental advancement rather than final success alone.

60 bilingual multimodal tasks inside real command-line harnesses in Docker, on which the best of 19 models reaches 62.2% and swapping the harness alone moves one model by up to 18 points.

Evaluates browser agents on 153 write-heavy tasks across 144 live websites, with final-request interception and synchronized video, screenshot, network, action, and message traces for auditable scoring and failure analysis.

MemoryAgentBench recasts long-context data into incremental multi-turn interactions testing accurate retrieval, test-time learning, long-range understanding, and selective forgetting; no agent class masters all four.

629 trajectories (523 risky, 106 safe) averaging 33 steps that audit risk arising under benign conditions; strong models detect trajectory-level risk but fall below 35 Strict-F1 on locating the risky step.

150 OSWorld-based tasks spanning deliberate misuse, prompt injection, and model misbehavior, with a judge agreeing with human annotation at 0.76 and 0.79 F1.

569 multi-turn records across 27 risk scenarios and 10 risk types, on which GPT-4o reaches 74.42% while no other evaluated model significantly beats random.

Compares metadata, OpenTelemetry-compatible, and OpenInference-compatible views, which retain 99.5% to 100% detection F1 while holding fault-origin step accuracy at or below 0.5%.

3,364 conversations across 48 scenarios in 12 regulated domains; each row gives an assistant a workplace rule and 2 to 5 options and records which options comply, whether the rule binds, which of nine pressures applies, and whether a directive is present.

Red-teams 10 agent benchmarks for reward-hacking exploits, surfacing 219 distinct flaws across eight recurring classes and scoring near-perfect on most benchmarks without solving a single task, then patches four of them below a 10% hackable-task ratio and fully repairs WebArena and OSWorld in three iterations.

1,302 web agent trajectories drawn from 5 benchmarks and 4 language models, each reviewed by an expert for success, side effects, and repetitiveness, used to score 12 language-model judges and to show that the rule-based evaluation of common benchmarks underreports agent success.

Training-free memory-augmented framework that lifts language-model safety evaluators toward human-expert accuracy, released with ASSEBench: 2,293 annotated records, 15 risk types, and 29 scenarios.

Separates visible validation tests from held-out compositional tests across 30 systems-level tasks, finding the pass-rate gap grows 28 percentage points per tenfold increase in code size.

Critique of agent benchmarking practice, arguing for cost-aware evaluation, adequate holdout sets, and reproducibility alongside accuracy.

self-hostable platform for tracing LLM and agent calls, running evaluations, managing prompts, and tracking cost and latency, with OpenTelemetry, LangChain, and OpenAI SDK integrations. MIT-licensed core, 2023-present.

open-source observability and evaluation tool built on OpenTelemetry for tracing, evaluating, and debugging LLM and agent applications, with auto-instrumentation for common frameworks. Elastic License 2.0, 2022-present.

a set of OpenTelemetry-compatible conventions and instrumentation packages for capturing traces from LLM and agent frameworks; exports traces to OpenTelemetry-compatible backends. Apache-2.0, 2023-present.

open-source platform for tracing, evaluating, and monitoring LLM applications and agentic workflows, with LLM-as-a-judge scoring, experiment tracking, and production dashboards. Apache-2.0, 2024-present.

OpenTelemetry-based instrumentation for LLM applications that emits standard OpenTelemetry traces routable to existing observability backends; SDKs across Python, TypeScript, Go, and Ruby. Apache-2.0, 2023-present.

open-source observability platform and AI gateway that logs LLM and agent traces, costs, and latency through a proxy and a unified API across providers. Apache-2.0, 2023-present.

OpenAI-compatible gateway that routes across providers with configurable failover, enforces budget and rate limits, executes MCP tools, and emits Prometheus metrics, OpenTelemetry traces, and per-request logs of prompt, response, token, cost, and latency. Apache-2.0, 2025-present.

open-source AI control plane that connects agents to MCPs, enforces access policies, and records searchable tool calls, permission changes, and access events. AGPL-3.0, 2025-present.

open-source evaluation framework that runs LLM and agent tests in a pytest-style workflow, with metrics for hallucination, relevancy, and task correctness that can be included in CI checks. Apache-2.0, 2023-present.

open-source framework to evaluate, test, and monitor ML and LLM systems, with 100+ built-in metrics spanning tabular data quality, data drift, and LLM judges. Apache-2.0, 2020-present.

open-source SDK for monitoring AI agents, with session replays, cost tracking, and failure detection; integrates with CrewAI, OpenAI Agents SDK, LangChain, and AG2 (formerly AutoGen). MIT-licensed, 2023-present.

framework-agnostic platform from the LangChain team for tracing, offline and online evaluation, and monitoring of LLM and agent applications. Commercial product, not open source; self-hosted and hybrid deployment are available on the Enterprise plan, 2023-present.

evaluation library for LLM applications, with RAG metrics such as faithfulness, answer relevancy, and context precision, agent metrics such as tool-call accuracy and topic adherence, and test set generation. Apache-2.0, 2023-2026 (last commit 2026-02-24, latest release v0.4.3 on 2026-01-13).

closes the debugging loop as detect, attribute, recover, and rerun, reaching 28.8% exact agent-and-step accuracy on Who&When with a qwen3.5-9b backbone against 21.7% for the strongest single-pass baseline and repairing 13 of 73 failed GAIA tasks in one rerun. MIT-licensed, 2026-present.

summarizes, searches, and clusters completed agent transcripts to surface broken tasks; adding packages that Docent revealed as missing raised the GPT-4o InterCode solve rate from 68.6% to 78% across 79 tasks. Apache-2.0, 2025-present.

OpenTelemetry-native agent observability with Signals that track a behavior described in plain English, such as an agent stuck in a loop, and alert on it across runs, plus SQL queries over span data. Apache-2.0, 2024-present.

OpenTelemetry-native evaluation recording latency, tokens, and cost per step, with seven agent-specific evaluators covering plan adherence, plan quality, tool selection, tool calling, execution efficiency, logical consistency, and external tool reliability. MIT-licensed, 2020-present.

runs AI-generated code in isolated cloud sandboxes so an unsafe action is contained rather than executed on the host, with Terraform-based self-hosting. Apache-2.0, 2023-present.

self-hosted microVM runtime giving agent code hardware-level isolation at an average boot under 100 milliseconds, spawned in-process with no daemon, with SDKs in five languages. Apache-2.0, 2024-present.

generates adversarial test suites from a plain-language description of the agent across OWASP LLM Top-10 threat categories, with multi-turn agent testing in v3. Apache-2.0, 2022-present.

deterministic runtime-conformance harness for the OpenAI Agents SDK that drives the real Runner through scripted responses on the public Model interface, and writes content-addressed evidence that can be rechecked without rerunning the SDK; built-in scenarios make no model API calls. MIT-licensed, 2026-present.

end-to-end agent auditing engine that uses the Agent Task Protocol to integrate tasks with different harnesses, automatically captures standardized execution traces, and assesses execution efficiency, tool use, task planning, and error recovery with multidimensional metrics. MIT-licensed, 2026-present.

GitHub Action and CLI that scores an agent specification against risk-tiered profiles, fails a release on unresolved blockers, and runs eight prompt-injection fixtures through LangGraph and OpenAI Agents adapters that retain node-linked traces and machine-readable results. MIT-licensed, 2026-present.

open-source behavioral testing tool for tool-using AI agents that simulates declared tool execution while evaluating tool calls, failures, retries, confirmations, destructive actions, action sequencing, and fabricated success, with integrations for OpenAI Agents SDK, PydanticAI, and custom Python agents. Apache-2.0, 2026-present.

records a coding-agent run below the harness, placing model calls, per-turn workspace changes, and opt-in MCP frames on one timeline. It then re-executes the agent against the recorded model responses and reports divergence, or forks the run from a filesystem checkpoint onto a different model for comparison. Apache-2.0, 2026-present.

zero-dependency provenance primitive that propagates mock or low-confidence taint through derived values, paired with five Claude Code skills that audit a repository for unverified claims and source-truth gaps. Apache-2.0, 2026-present.

evaluation layer for coding-agent runs that executes each candidate in an isolated workspace and retains hash-linked execution receipts, post-run Git manifests, and evaluator configuration for later inspection. MIT-licensed, 2026-present.

Proposes short standardized documents that report a model's intended use, evaluation conditions, and performance across groups as accountability artifacts for deployed models.

Introduces a standard datasheet for documenting a dataset's motivation, composition, collection process, and recommended uses to improve transparency and accountability.

Defines SMACTR, an internal audit framework that produces a documented decision trail across the development lifecycle so teams can assess systems before deployment.

Argues that meaningful third-party AI audits need more than query access, comparing black-box, white-box, and outside-the-box methods and their accountability implications.

Extends OAuth 2.0 and OpenID Connect with agent-specific credentials to scope permissions delegated to autonomous agents and link those permissions to a responsible human.

Surveys current identity, delegation, and accountability standards for agents acting across organizational boundaries and maps the open gaps that current frameworks leave unaddressed.

JSON-RPC 2.0 protocol that standardizes how applications expose tools, resources, and prompts to language models, with an optional OAuth 2.1 authorization profile requiring Protected Resource Metadata discovery (RFC 9728), resource-indicator audience binding (RFC 8707), and issuer validation (RFC 9207) on HTTP transports. Agentic AI Foundation (a directed fund under the Linux Foundation), revision 2026-07-28.

task delegation between independent agents, where each server publishes an Agent Card declaring identity, capabilities, skills, endpoint, and authentication across API key, HTTP, OAuth 2.0, OpenID Connect, and mutual TLS. Linux Foundation, contributed by Google, v1.0.1 (28 May 2026).

users sign Checkout and Payment Mandates as verifiable credentials, with an open stage carrying spending constraints and a closed stage carrying a specific authorization, so each agent-initiated transaction leaves a non-repudiable cryptographic record. FIDO Alliance, donated by Google, v0.2.

vendor-neutral telemetry schema for GenAI and agent spans, metrics, and events, covering model calls, tool calls, token usage, and agent and framework operations to support comparison of traces across stacks. OpenTelemetry / CNCF; all gen_ai.* attributes were deprecated in the core semantic-conventions repository at v1.42.0 (12 June 2026) and moved to this dedicated repository.

cross-sector companion to the NIST AI RMF that names twelve GenAI risk areas and over 200 suggested actions across the Govern, Map, Measure, and Manage functions. NIST, July 2024.

legal duty requiring high-risk AI systems to record events throughout their lifetime for risk identification, post-market monitoring, and operational oversight. Article 26(6) requires deployers to retain the logs for at least six months. EU, 2024; the Digital Omnibus on AI (Regulation (EU) 2026/1744, in force 27 July 2026) moved high-risk application to 2 December 2027 for Annex III areas and 2 August 2028 for AI embedded in regulated products.

first certifiable AI management system standard, setting requirements to establish, operate, and improve governance controls across the AI lifecycle, including risk assessment, impact assessment, and performance measurement. ISO/IEC JTC 1/SC 42, 2023.

living knowledge base of adversary tactics and techniques against AI systems, modeled on MITRE ATT&CK and built from real-world attacks and red-team demonstrations. MITRE, v2026.07 (7 August 2026): 16 tactics, 101 techniques, 77 sub-techniques, 37 mitigations, and 68 case studies, adding three AI Agent Tool Poisoning sub-techniques (AML.T0110.000/.001/.002) and AML.T0115.

community-maintained risk lists for language-model and agentic applications, covering prompt injection, excessive agency, tool misuse, memory and context poisoning, and goal hijacking, with a builder-side companion guide of technical recommendations. OWASP, 2025 (LLM) / 2026 (Agentic).

base-level weakness entry cataloging prompt injection as a mappable software weakness, with a LangChain tool-agent demonstrative example. MITRE CWE 4.20, mapping status ALLOWED.

seven-layer threat-modeling method a builder can run against an agent architecture. Cloud Security Alliance, 2025; a single-author blog publication rather than a ratified standard, so it sits at a lower evidentiary register than the NIST, ISO, MITRE, and OWASP entries above.

independent governance proposal that organizes agent oversight into five layers, specifies evidence-collection and integrity controls, grades the evidence and design inferences behind its mechanisms, and runs a public evidence-correction process. ZhangRui987, 2026; a proposed design rather than a ratified standard or an evaluated implementation, so it sits at the same lower evidentiary register as MAESTRO above.

cryptographically signed, tamper-evident metadata standard that records the origin and edit history of media, including a manifest for AI-generated and AI-edited content. C2PA (Adobe, Microsoft, BBC, Intel, Truepic, Sony, and others), v2.2 (2025).

append-only Merkle tree log with inclusion proofs, showing an entry is in the log, and consistency proofs, showing the log was only appended to; the origin of the verifiable-log pattern that tamper-evident agent trails reuse. IETF, 2013 (v1) / 2021 (v2).

signed, machine-readable statements binding a subject artifact to a predicate describing what was done to it, giving a common envelope for provenance claims across steps and tools. in-toto / CNCF, 2018-2026 (living spec).

graded framework defining what provenance a build must produce and how tamper-resistant it must be, with each level naming concrete requirements rather than intent. OpenSSF, v1.0 (2023).

signing envelope that authenticates a payload together with its type, removing the canonicalization ambiguity that lets a signature be replayed against a different interpretation of the same bytes; used by in-toto and Sigstore. Secure Systems Lab, 2021-2026.

transparency log service for signed software artifacts and attestations, built on an append-only Merkle log and serving inclusion and consistency proofs over a public API. Sigstore / OpenSSF, Apache-2.0, 2020-present.