Skip to content

vaab/ai-audit

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

78 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ai-audit

Description

CLI tool to audit and monitor AI assistant sessions. Supports Claude Code, OpenCode, and pi (badlogic/pi-mono).

Installation

cargo install --path .

This installs the ai-audit binary to ~/.cargo/bin/.

Usage

Command structure

The CLI uses a <noun> <verb> structure. All session-scoped commands live under ai-audit session <verb>. Cross-session concerns (activity, token-usage, rate) stay at the top level.

Top-level commands

CommandDescription
session <verb>Session inspection and management (see below)
activity <list/get>User activity tracking (messages + permissions)
token-usage <timespan>Per-message LLM token consumption events
rate <instruction>Rate agent instructions against test cases

Session subcommands

Short aliases and prefix inference are supported:

CommandAliasesPrefix examples
session listlss l, s li
session currentcurs c
session previousprevs pr
session transcripttrs t
session permissionspermss pe
session usagetokenss u
session assisted-bys a
session infos i
session nudges n
session deletes d

So ai-audit s ls is equivalent to ai-audit session list. Note that s p is ambiguous between permissions and previous — clap prints an error listing the candidates. Use s pe or s pr to disambiguate.

The legacy top-level forms (list-sessions, current-session, last-session, transcript, permissions, usage, assisted-by) still work for backwards compatibility but print a deprecation warning on stderr. last-session is now spelled session previous (or s prev).

List Sessions

List all available sessions from Claude Code, OpenCode, and pi:

# List all sessions (chronological order)
ai-audit session list

# Filter by type
ai-audit session list -t claudecode
ai-audit session list -t opencode
ai-audit session list -t pi

# Short form (alias)
ai-audit s ls

# JSON output
ai-audit session list --json

Output format (tab-separated):

2026-01-19T14:18:18.363+00:00	ses_42964fa7effeVBExFbcGLmC6Hv	opencode
2026-01-20T00:42:31.010+00:00	5593c91e-7bcd-4dff-b52f-f6266899f2d4	claudecode

Search Sessions

Filter sessions using text, time, and project filters:

# Search by content (all messages)
ai-audit session list --search "some keyword"

# Filter by time range
ai-audit session list --timespan today
ai-audit session list --timespan "2026-02-01..2026-02-11"

# Filter by project directory
ai-audit session list --project .
ai-audit session list -p ../fyl

# Combine filters
ai-audit session list --timespan today --search "keyword" -t opencode

Status filters (OpenCode)

session list and session usage also accept a status-filtering subsystem derived from each session’s last message and its parts:

# Filter by static status (comma-separated)
ai-audit session list --status assistant-empty,assistant-tool-stuck

# Shorthand for the resumable static-status set
ai-audit session list --resumable

# Filter by when the last message landed
ai-audit session list --last-message-in 30m

# Include / filter by live status (probes the running opencode server)
ai-audit session list --output-live-status
ai-audit session list --filter-by-live-status running,idle
ai-audit session list --server-url http://127.0.0.1:4096

Static-status values: completed, user-pending, assistant-empty, assistant-partial, assistant-tool-stuck. Live-status values: running, idle, server-unreachable.

Current Session Detection

Detect which AI session is running (or was running) in the current context:

# Auto-detect (env vars, process tree, fingerprinting)
ai-audit session current

# Identify session by matching text in last N messages
# (useful after tmux-resurrect to find which session was active)
ai-audit session current --match "visible text from pane"
ai-audit session current --match "text" -t opencode -n 3

# Identify session from a PID (e.g., an opencode process)
ai-audit session current --pid 286943
ai-audit session current --pid 286943 -t opencode

# Output formats
ai-audit session current --json
ai-audit session current -0

# Detect the last session used in the current tmux pane
# (was: ai-audit last-session)
ai-audit session previous
ai-audit s prev -t opencode

Detection strategies (mutually exclusive):

ModeFlagStrategy
Auto(default)Env vars → process tree → CWD + fingerprint
Text match--matchSearch last N messages for the given string
PID--pidRead /proc/<pid>/ (env, cwd, provider detection)

Options:

FlagDescription
-m, --matchText to match in recent messages
--pidPID of an AI assistant process
-t, --typeFilter by provider (claudecode, opencode, pi)
-n, --last-messagesMessages to search with --match (default: 5)
-p, --projectFilter by project path

Transcript

Display the full conversation transcript of a session:

# With explicit session ID
ai-audit session transcript <session-id>

# Auto-detect current session
ai-audit session transcript

# Short form (alias)
ai-audit s tr

# Show only the last N entries
ai-audit session transcript -n 20

# Include thinking/reasoning blocks
ai-audit session transcript -v

Permissions

List permission events for a session (claudecode or opencode):

# Claude Code session (UUID format)
ai-audit session permissions 5593c91e-7bcd-4dff-b52f-f6266899f2d4

# OpenCode session (ses_ prefix)
ai-audit session permissions ses_426e8c1ceffecECQZeKXvtWNoo

# Short form (alias)
ai-audit s perms ses_426e8c1ceffecECQZeKXvtWNoo

OpenCode output shows tool calls with permission decisions:

2026-01-20 01:49:03      ASK      bash         axdfguij

pi has no permission/approval model, so session permissions errors out for pi sessions.

Session Info

Show metadata for a single session (auto-detects the current session when no ID is given):

# Explicit session ID
ai-audit session info ses_426e8c1ceffecECQZeKXvtWNoo

# Auto-detect the current session
ai-audit session info

# Skip the live-status HTTP probe (OpenCode only; useful offline)
ai-audit session info --no-live

# JSON output
ai-audit session info --json

For OpenCode sessions the live status is probed from the running opencode HTTP server unless --no-live is passed.

Nudge

Revive resumable OpenCode sessions by replaying or appending a prompt (OpenCode only):

# Nudge a single session
ai-audit session nudge ses_426e8c1ceffecECQZeKXvtWNoo

# Nudge every session matching filters
ai-audit session nudge --all --status assistant-empty
ai-audit session nudge --all -p . --search "stuck"

# Preview the plan without posting any prompt
ai-audit session nudge --all --dry-run

# Override the prompt that gets posted (default: "continue")
ai-audit session nudge ses_x --continue-prompt "please continue"

# Fork instead of mutating the original session
ai-audit session nudge ses_x --fork

A positional <session-id> cannot be combined with the filter flags (--project, --search, --timespan, --last-message-in, --status); use --all with filters to target a batch.

Delete

Wipe one or more sessions across every storage location ai-audit reads (transcripts, debug logs, SQLite rows, legacy file-tree, session-index cache):

# Delete a single session
ai-audit session delete ses_426e8c1ceffecECQZeKXvtWNoo

# Preview what would be deleted (the only safety mechanism)
ai-audit session delete --all -t opencode --dry-run

# Batch delete via filters (--all required as a guard)
ai-audit session delete --all --timespan "..2026-01-01"

# Pipe IDs from `session list -j` (or `-0`)
ai-audit session list -j --status assistant-empty \
  | ai-audit session delete --ids-file -

# Also delete child sessions of a parent
ai-audit session delete ses_parent --cascade

Safety: there is no confirmation prompt and no --yes--dry-run is the only safeguard, so the filter is the contract. Deleting the current session (matched via $*_SESSION_ID or tmux fingerprinting) is rejected. Deleting a parent session is refused unless --cascade is passed.

Token Usage

Per-message LLM token consumption events over a timespan, across all providers (claudecode, opencode, pi). One record per assistant message that consumed tokens.

# All token-consuming events today, default human columns
ai-audit token-usage today

# With column header row
ai-audit token-usage today --header

# JSON Lines (machine-parseable, all 16 fields)
ai-audit token-usage today -j

# NUL-separated records (xargs -0 friendly, all 16 fields)
ai-audit token-usage today -0

# Filter by project (basename of nearest .git ancestor)
ai-audit token-usage today -p ai-audit

# Sessions outside any git repo
ai-audit token-usage today -p ""

# Filter by harness, LLM provider id, or model substring
ai-audit token-usage today -t pi --provider-id anthropic
ai-audit token-usage 30m --model haiku

# Restrict to specific sessions (repeatable)
ai-audit token-usage today -s ses_abc -s ses_def

# Custom field set (also repeatable)
ai-audit token-usage today -f timestamp,project,total --header

Available fields

For --fields:

NameDescription
timestampMessage creation time (local in human, UTC float in -0/-j)
session_idFull session id
providerHarness: claudecode / opencode / pi
provider_idLLM provider id (anthropic, openai-codex, …)
modelModel id (e.g. claude-sonnet-4-5)
cwdFull session working directory
project_pathParent of .git ancestor (with $HOME~); cwd if no .git
projectBasename of .git ancestor; empty if no .git
subpathcwd relative to .git ancestor; empty if cwd == ancestor
inputInput tokens
outputOutput tokens
cache_readTokens read from cache
cache_writeTokens written to cache (OpenCode-specific)
cache_creationCache-creation input tokens (Claude Code-specific)
reasoningReasoning/thinking tokens
totalSum of all token fields

Default human field set: timestamp, project, session_id, model, input, output, cache_read, total

NUL/JSON without --fields emit all 16 fields per the cli-guidelines “complete info” rule for machine-parseable formats.

Project derivation

Given a session whose cwd is /home/vaab/dev/rs/ai-audit/doc, with a .git directory at /home/vaab/dev/rs/ai-audit/:

FieldValue
cwd/home/vaab/dev/rs/ai-audit/doc
project_path~/dev/rs
projectai-audit
subpathdoc

When no .git ancestor exists, project and subpath are empty, project_path falls back to cwd (with $HOME collapse), and cwd is unchanged. Use --project””= to filter for these.

Activity

User activity tracking — messages and permission grants — keyed by an activity identifier (e.g. claude-msg@rs/ai-audit, pi-msg@rs/ai-audit).

# List all available activity identifiers
ai-audit activity list

# Get activity events for a timespan
ai-audit activity get today
ai-audit activity get "2026-02-01..2026-02-11"

# Filter by identifier(s)
ai-audit activity get today claude-msg@rs/ai-audit

# Restrict to specific sessions (repeatable)
ai-audit activity get today -s ses_abc -s ses_def

# Read additional identifiers from a NUL-separated file (or `-` for stdin)
ai-audit activity get today --categs-file idents.nul

# Machine-parseable output
ai-audit activity get today -j
ai-audit activity get today -0

Since pi has no permission model, only pi-msg@<project> activity identifiers are emitted for pi sessions.

Rate

Rate agent instructions against test cases. Runs the agent under test (via pi) with each requested model, then judges the output against the test’s checklist (also via pi).

# Rate an instruction file against a test file or directory
ai-audit rate instructions.md --test tests/

# Override agent / judge models (comma-separated)
ai-audit rate instructions.md --test tests/ \
  --agent-models anthropic/claude-sonnet-4 \
  --judge-models anthropic/claude-opus-4

# Set a per-invocation timeout and bypass the cache
ai-audit rate instructions.md --test tests/ --timeout 120 --no-cache

# Supply a custom judge prompt
ai-audit rate instructions.md --test tests/ --judge-prompt judge.md
FlagDescription
--testTest file or directory (required)
--agent-modelsAgent models, comma-separated (default: pi default)
--judge-modelsJudge models, comma-separated (default: pi default)
--timeoutTimeout in seconds
--no-cacheForce recomputation, ignore cache
--judge-promptPath to a custom judge prompt

Output Formats

Human-readable (default)

2026-01-15 00:26:31      INIT         userSettings (41 pre-existing rules loaded)
2026-01-15 00:28:16      REQUEST      Bash: ls -la ~/.claude/
2026-01-15 00:28:22      GRANTED      [session] Read(//home/user/work/**)
2026-01-15 00:32:41      DIR_ACCESS   [session] dirs: /home/user/project
2026-01-15 00:32:41      MODE         mode -> acceptEdits
2026-01-15 01:34:57      REQUEST      Edit: /home/user/project/src/main.rs
2026-01-15 01:35:09      GRANTED      [session] Read(//home/user/.claude/**)

REQUEST events show the tool name and relevant details (command for Bash, file path for Read/Write/Edit, pattern for Glob/Grep) when available.

NUL-separated (-0)

Tab-delimited fields, NUL-terminated records.

FieldDescription
timestampUTC float seconds since epoch
typeREQUEST, GRANTED, DIR_ACCESS, MODE
detailsType-specific (see below)

Details format by type:

  • REQUEST: tool\tcommand_or_path (e.g., Bash\tls -la)
  • GRANTED: destination\trules (comma-separated)
  • DIR_ACCESS: destination\tdirectories (comma-separated)
  • MODE: mode_value

NDJSON (-j)

One JSON object per line. Fields by event type:

FieldTypesDescription
timestampallUTC float seconds since epoch
typeallrequest/granted/dir_access/mode_change
toolrequestTool name (Glob, Grep, Bash, etc.)
inputrequest (optional)Full tool input object (command, file_path, etc.)
destinationgranted, dir_accesssession/localSettings/userSettings
rulesgrantedArray of permission rules
directoriesdir_accessArray of directory paths
modemode_changeNew mode value

License

MIT

About

CLI tool to audit and monitor AI assistant sessions (Claude Code, OpenCode, pi)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages