Shiro is a SQL fuzzing CLI for TiDB that targets optimizer bugs. It generates random schemas and SQL (DDL/DML/SELECT) and validates correctness using multiple oracles inspired by SQLancer and recent research papers.

• Random schema + data generation
• Weighted feature toggles (joins, CTEs, subqueries, aggregates, plan cache)
• Oracles: NoREC, TLP, DQP, CERT, CODDTest, DQE
• Query Plan Guidance (QPG) for plan-diversity-driven state mutations
• Panic/crash detection with automatic PLAN REPLAYER DUMP and artifact download
• Case reports with SQL, schema, and bounded data samples
• Optional DDL coverage for indexes, views, check constraints, foreign keys, and partitioned tables

1. Start TiDB (nightly or latest stable)
2. Configure config.yaml as needed
3. Run:

go run ./cmd/shiro -config config.yaml

Shiro prints the final resolved configuration at startup.

Reports are written under reports/ by default.

Shiro can record CI runtime metadata directly from environment variables and persist it into each case summary.json as run_info.

Preferred explicit variables:

• SHIRO_CI
• SHIRO_CI_PROVIDER
• SHIRO_CI_REPOSITORY
• SHIRO_CI_BRANCH
• SHIRO_CI_COMMIT
• SHIRO_CI_WORKFLOW
• SHIRO_CI_JOB
• SHIRO_CI_RUN_ID
• SHIRO_CI_RUN_NUMBER
• SHIRO_CI_EVENT
• SHIRO_CI_PULL_REQUEST
• SHIRO_CI_ACTOR
• SHIRO_CI_BUILD_URL

If explicit SHIRO_CI_* values are not set, Shiro auto-detects common CI providers and consumes defaults (for example, GitHub Actions GITHUB_* variables).

Set workers in config.yaml. Each worker runs in its own database (<database>_wN) to keep session variables isolated.

Every report_interval_seconds, Shiro logs the ratio of parser-valid SQL to total SQL observed in that interval. When QPG is enabled and logging.verbose is true, it also prints per-interval QPG coverage deltas (plans/shapes/ops/join types). Set logging.log_file to write detailed logs to a file (default logs/shiro.log), while stdout keeps only the basic interval summaries and errors. Stdout entries are also mirrored into the log file.

features.not_exists and features.not_in toggle negation forms, while weights.features.not_exists_prob and weights.features.not_in_prob control how often NOT EXISTS/NOT IN are generated.

oracles.strict_predicates: true (default) limits TLP/CODDTest to simple deterministic predicates to reduce false positives. Set it to false if you want broader coverage at the cost of more noisy cases.

oracles.coddtest_case_when_max (default 2) caps dependent CODDTest CASE WHEN branches so rewritten predicates do not become excessively large.

DQP now includes SET_VAR(tidb_opt_partial_ordered_index_for_topn='COST'|'DISABLE') and join-path SET_VAR(tidb_allow_mpp=ON|OFF) in its built-in SET_VAR candidates. You can also inject extra DQP hints from config via oracles.dqp_external_hints. The DQP complexity guard for set_ops + derived_tables is configurable via oracles.dqp_complexity_set_ops_threshold and oracles.dqp_complexity_derived_threshold (defaults 2/4), and is evaluated during query generation so DQP can retry candidates before final skip classification. EET also applies a unified table-factor budget via oracles.eet_complexity_join_tables_threshold (default 5), counting main query table factors plus CTE definitions and CTE-body table factors. When MPP is enabled (mpp.enable: true), Shiro normalizes mpp.tiflash_replica to at least 1 and issues ALTER TABLE ... SET TIFLASH REPLICA <n> after each base-table creation, then waits (100ms polling, 2m timeout) until SELECT COUNT(*) FROM information_schema.tiflash_replica WHERE AVAILABLE=0 becomes 0. To globally disable Shiro-managed MPP exploration, set mpp.enable: false; this disables TiFlash replica provisioning and removes DQP MPP SET_VAR hints (tidb_allow_mpp, tidb_enforce_mpp) from built-in/external candidates. Legacy oracle-level keys (oracles.disable_mpp, oracles.mpp_tiflash_replica) are still accepted for compatibility.

Each entry can be either:

• a full optimizer hint, for example HASH_JOIN(t1, t2) or SET_VAR(tidb_opt_use_toja=OFF)
• a raw var=value assignment, which Shiro wraps as SET_VAR(var=value)

Example:

mpp:
  enable: false
  tiflash_replica: 1
oracles:
  dqp_complexity_set_ops_threshold: 2
  dqp_complexity_derived_threshold: 4
  eet_complexity_join_tables_threshold: 5
  dqp_external_hints:
    - "SET_VAR(tidb_opt_partial_ordered_index_for_topn='COST')"
    - "tidb_opt_partial_ordered_index_for_topn='DISABLE'"
    - "tidb_enforce_mpp=ON"

oracles.groundtruth_max_rows caps per-table sample size used by the GroundTruth join-count checker (default 50). Lower values reduce runtime overhead but may increase false negatives.

Enable adaptive.enabled to let Shiro adjust selection of actions/oracles/DML based on bug yield. By default, only oracle selection adapts when adaptive.enabled is true; set adaptive.adapt_actions, adaptive.adapt_dml, or adaptive.adapt_features to include them. QPG works alongside bandits: bandit weights are applied first, then QPG can temporarily override join/subquery/aggregate weights when plan coverage stalls (TTL-based).

Enable qpg.enabled to collect EXPLAIN plan signatures. When a repeated plan is observed, Shiro can mutate the database state (index/analyze) to explore new plans. Configure qpg.explain_format (default brief), qpg.mutation_prob (0-100), and the qpg.seen_sql_* cache controls. Default QPG cache values are tuned for longer runs: seen_sql_ttl_seconds=120, seen_sql_max=8192, seen_sql_sweep_seconds=600. QPG also tracks operator/shape coverage to temporarily boost join/aggregate/subquery generation when coverage stalls. To reduce overhead, QPG caches recent SQL strings and skips EXPLAIN for repeated queries within a short window. When EXPLAIN FORMAT='json' is used, QPG extracts operator IDs from the JSON to continue coverage tracking. QPG also tracks operator-sequence signatures (plan operator lists) to nudge aggregate/subquery coverage when operator diversity stalls. JSON parsing accepts either id or operator keys and normalizes operator names for coverage accounting. QPG normalizes EXPLAIN text (table/column/index tokens, numeric literals) before hashing to reduce noise.

For short CI runs, you can tighten QPG trigger thresholds and keep override TTLs short:

qpg:
  enabled: true
  explain_format: "brief"
  mutation_prob: 35
  no_join_threshold: 2
  no_agg_threshold: 2
  no_new_plan_threshold: 3
  no_new_op_sig_threshold: 3
  no_new_shape_threshold: 3
  no_new_join_type_threshold: 2
  no_new_join_order_threshold: 2
  override_ttl: 3
  template_override:
    no_new_join_order_threshold: 2
    no_new_shape_threshold: 3
    no_agg_threshold: 2
    no_new_plan_threshold: 3
    join_weight_boost: 7
    agg_weight_boost: 7
    semi_weight_boost: 6
    enabled_prob: 65
    override_ttl: 3

Set plan_cache_only: true for a focused plan-cache run that executes only prepared statements. In normal mode, Shiro still runs prepared statements and applies the same plan-cache checks; this flag just isolates that workflow. When plan_cache_only is enabled, runner skips TiFlash replica provisioning (ALTER TABLE ... SET TIFLASH REPLICA) during initial schema setup. Prepared plan-cache paths force tidb_allow_mpp=OFF (and try tidb_enforce_mpp=OFF) on the session to avoid MPP-plan noise in cache-hit checks. The plan-cache check verifies SELECT @@last_plan_from_cache = 1 on the second execution (when no warning indicates a cache skip). On a detected bug, the runner switches to a fresh database (<database>_rN) and reinitializes schema/data. Plan-cache-only cases now record the exact PREPARE/EXECUTE SQL and parameter values in the case files. Signature comparisons round floating-point outputs to reduce false positives; set signature.round_scale and signature.plan_cache_round_scale to tune.

Enable minimize.enabled to shrink captured cases. Shiro attempts to remove redundant INSERTs and SQL statements while rechecking the failure in a fresh database. Tune minimize.max_rounds to cap delta-debugging passes and minimize.timeout_seconds to bound minimization time (defaults are more aggressive to allow deeper shrinking). Set minimize.merge_inserts to re-merge single-row inserts into multi-row batches after reduction for smaller output files. Minimized outputs are saved as case_min.sql, inserts_min.sql, and repro_min.sql alongside the original files.

For error_reason=pqs:runtime_1105, report classification keeps the runtime bug signal (bug_hint=tidb:runtime_error) regardless of minimize result, and adds reproducibility metadata for triage:

• runtime_bug_reproducible=true|false (true only when minimize_status=success)
• runtime_bug_hint_gated=true for non-success statuses (skipped, disabled, not_applicable, etc.)
• runtime_bug_hint_gate_reason=requires_repro by default for non-success statuses (preserved if already pre-filled by upstream logic)

Generate a JSON report that a static frontend can consume:

go run ./cmd/shiro-report -input reports -output web/public

cmd/shiro-report now defaults to reading .report; pass -input when your run output directory is different (for example the default runner output reports/).

For GCS inputs, provide a config with storage.gcs enabled (legacy s3:// inputs still work with storage.s3):

go run ./cmd/shiro-report -input gs://my-bucket/shiro-reports/ -config config.yaml -output web/public

cd web
npm install
npm run build

Or run both steps with:

make report-web

Deploy the web/out/ directory (GitHub Pages/Vercel). The frontend reads reports.json (fallback report.json) at runtime, so you only need to update the JSON to refresh the view. Set NEXT_PUBLIC_REPORTS_BASE_URL to point the frontend at a public bucket or CDN when hosting JSON outside the app bundle.

The report JSON now includes plan_signature (QPG EXPLAIN hash) and plan_signature_format (plain/json); the UI can filter by both. Each case entry also includes case_id, archive_name, archive_codec, archive_url, and report_url.

To publish report manifests to an S3-compatible endpoint (for example Cloudflare R2) and sync metadata to Cloudflare Worker + D1:

go run ./cmd/shiro-report \
  -input s3://my-bucket/shiro-reports/ \
  -config config.yaml \
  -output web/public \
  -artifact-public-base-url https://<artifact-public-domain> \
  -publish-endpoint https://<accountid>.r2.cloudflarestorage.com \
  -publish-region auto \
  -publish-bucket <r2-bucket> \
  -publish-prefix shiro/manifests/latest \
  -publish-access-key-id <r2-access-key> \
  -publish-secret-access-key <r2-secret-key> \
  -publish-public-base-url https://<r2-public-domain> \
  -worker-sync-endpoint https://<worker-domain>/api/v1/cases/sync \
  -worker-sync-token <worker-api-token>

When publish/sync flags are omitted, cmd/shiro-report keeps existing local behavior. When -artifact-public-base-url is not provided, per-case report_url and archive_url are only emitted when the source upload location is already HTTP(S). For GCS, -artifact-public-base-url should be the public HTTP base that serves your bucket (for example https://storage.googleapis.com/<bucket> or a CDN domain). To publish manifests to GCS, set -publish-gcs-bucket (and optionally -publish-gcs-prefix), and ensure GOOGLE_APPLICATION_CREDENTIALS is available for ADC. Cloudflare metadata/search worker code is under web/cloudflare-worker/.

At each report interval, Shiro writes dynamic_state.json in the working directory with bandit/QPG/feature weights so runs can be resumed or compared.

• If PLAN REPLAYER DUMP returns only a file name, set plan_replayer.download_url_template in config.yaml.
• Shiro uses PLAN REPLAYER DUMP EXPLAIN to avoid executing the query.
• TiDB returns a token (zip name). If the dump output does not include a URL, configure plan_replayer.download_url_template using your TiDB status port, e.g. http://127.0.0.1:10080/plan_replayer/dump/%s.
• The parser validation uses github.com/pingcap/tidb/pkg/parser only.
• Join chain length is capped by max_join_tables.

Configure storage.gcs in config.yaml. When enabled, each case is uploaded under UUID path (gs://<bucket>/<prefix>/<case_id>/...), and the summary includes upload_location.

Legacy S3-compatible uploads remain available through storage.s3, but new deployments should use GCS.

When neither storage.gcs.enabled nor storage.s3.enabled is true, Shiro keeps legacy local report layout (case_XXXX_uuid directories) and summary-only artifact flow.

If you still use the storage.s3 path in CI (for example with GCS HMAC interoperability), configure storage.s3 and map secrets/variables as follows.

Example static config:

storage:
  s3:
    enabled: true
    endpoint: https://storage.googleapis.com
    region: auto
    bucket: your-bucket
    prefix: shiro-reports
    use_path_style: false

Example GitHub Actions env injection:

env:
  AWS_ACCESS_KEY_ID: ${{ secrets.SHIRO_GCS_HMAC_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.SHIRO_GCS_HMAC_SECRET_ACCESS_KEY }}
  AWS_SESSION_TOKEN: ${{ secrets.SHIRO_GCS_SESSION_TOKEN }}