fix(common): contain jaq parser panics in jq::parse_file, and wire the fuzz dictionaries - #1209
Conversation
jaq/hifijson can panic on adversarial JSON — a fuzzer found an Ord total-order violation in std's sort on numbers that overflow f64 to ±inf, aborting the process. parse_file's input is supply-chain-influenced (upstream project data files, via decode::stacks), and the module already contracts to return Err on bad input, never to abort. Wrap both the JSON and TOML parse branches in catch_unwind and convert a caught unwind to JqError. Proven by parse_file_contains_jaq_panic, an ordinary #[test]: the jq_parse_json fuzz target builds panic=abort (libfuzzer-sys installs an abort hook) and so cannot observe the containment. Minimized crash fixture checked in. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dictionaries for the structured targets — graph wire tag bytes + JSON keys, minimal.toml section/key vocabulary, jq JSON tokens, and the ArgSchema grammar — steer the mutator toward structurally valid inputs instead of rediscovering framing from scratch. Measured overnight: graph_from_bytes edge coverage rose 6.5k → 8.3k with graph.dict live, and arg_schema_parse moved off a multi-billion-exec plateau. Seeds preserve both jq_parse_json Ord-violation crash variants for regression, alongside the checked-in minimized fixture. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The dictionaries landed as files nothing read. libFuzzer does not discover a dictionary on its own — it has to be handed `-dict=` — and neither the `just fuzz` recipe nor the docs passed one, so all four were inert. That is the same failure mode as a fuzz target that no longer compiles: it reads as coverage that is not there. Name each dict after the target it feeds (`args.dict` -> `arg_schema_parse.dict`, and so on) so the recipe can find it by convention, and pass `-dict=` when `fuzz/<target>.dict` exists. A target without one is unaffected; a dictionary only biases mutation. Verified: `just fuzz common jq_parse_json` now reports `Dictionary: 14 entries` where it previously reported none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 27 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (10)
Comment |
twitchyliquid64
left a comment
There was a problem hiding this comment.
Should we also report this upstream ?
|
Yes — and it turns out to be a clean upstream bug, so thanks for asking. I dug in. It is not our call site. Root cause is in (Self::Int(x), Self::Int(y)) => x.cmp(y), // exact
(Self::Int(i), Self::Float(f)) => float_cmp(*i as f64, *f), // lossy
(Self::BigInt(x), Self::BigInt(y)) => x.cmp(y), // exact
(Self::BigInt(x), Self::Float(y)) => float_cmp(x.to_f64().unwrap(), *y) // saturates to infSo equality is not transitive, and it needs nothing exotic to show: let a = Num::Int(1_000_000_000_000_000_001);
let b = Num::Int(1_000_000_000_000_000_002);
let f = Num::Float(1e18); // f64 spacing at 1e18 is 128
// a == f, b == f, yet a < bThe BigInt arm is the same defect with a wider mouth: every integer above Separately, Not reported upstream (checked the tracker; #232 is unrelated jq-compat stuff) and 2.0.2 is the latest release, so it is live. I have an issue drafted with both reproducers and a suggested fix — compare Int/BigInt against Float exactly, since every finite f64 is a dyadic rational, which is contract-correct and more accurate than today. Will file it against Worth being explicit that this does not change the case for this PR: |
|
Went back and checked the jaq tracker properly — my last comment said "not reported upstream" off a keyword search, which was too thin an answer to give you. Enumerating all 460 issues+PRs ( Still nothing tracking it, confirmed properly this time: no issue or PR mentioning total order, There is a sharper bug than the one I described. PR #343 ("Simplify integer-float equality", merged Oct 2025) gave // PartialEq (num.rs:336)
(BigInt(i), Float(f)) | (Float(f), BigInt(i)) => f.is_finite() && float_eq(i.to_f64().unwrap(), *f),
// Ord (num.rs:364)
(BigInt(x), Float(y)) => float_cmp(x.to_f64().unwrap(), *y),So #343 also saw this coming and priced it wrong, which is the most useful thing to tell an upstream maintainer. Its own description anticipates the collision — "if you store many very large integers that are quite close to each other and that fit into One correction to my last comment: I said the Worth noting for our own purposes: jaq does have fuzz targets (#262), but they cover jaq-core lexer/parser, not the jaq-json reader. That is the gap our Draft is rewritten and ordered by how unarguable each point is. Nothing here changes this PR — the guard is still the right call regardless of what upstream decides. |
|
going to merge and deal with the notification separately |
From the latest fuzzing bundle, rebased onto current main (base was 4 commits behind) with authorship normalised.
The fix
jaq/hifijsonpanics on adversarial JSON — anOrdtotal-order violation in std sort on numbers that overflow f64 to ±inf, which aborts the process.jq::parse_fileis supply-chain-influenced (upstream project data files, viadecode::stacks) and its contract is to returnErron bad input, never to abort. Both parse branches are now wrapped incatch_unwindand a caught unwind becomes aJqError.Verified two ways:
panic = "abort", socatch_unwindgenuinely applies to shipped builds rather than only to tests.user-provided comparison function does not correctly implement a total order. It also asserts the panic path specifically, so it cannot pass if the reproducer ever stops panicking.Worth noting the target could not prove this itself —
jq_parse_jsonbuildspanic = "abort"under libfuzzer-sys, so containment is invisible to it. An ordinary#[test]carries the proof instead.The dictionaries — and one gap I closed
The bundle added four libFuzzer dictionaries. As shipped they were dead files: libFuzzer never discovers a dictionary on its own, it has to be handed
-dict=, and neither thejust fuzzrecipe nor the docs passed one. Same failure mode as a fuzz target that stops compiling — it reads as coverage that is not there.So each dict is now named after the target it feeds (
args.dict->arg_schema_parse.dict, etc.) and the recipe passes-dict=whenfuzz/<target>.dictexists. Convention does the wiring, so the next dict needs no recipe change.Verified:
just fuzz common jq_parse_jsonreportsDictionary: 14 entries, where it previously reported none.Also ships the two minimised jq crash inputs as regression seeds, and updates docs/fuzzing.md (which still listed dictionaries as a future idea).
cargo test -p common51 passing, clippy and fmt clean.Note
Contain jaq parser panics in
jq::parse_fileand wire fuzz dictionariesjq::parse_filewith a newguardfunction that usescatch_unwindto catch panics from the jaq/hifijson stack, converting them into structuredJqErrorvalues instead of unwinding..unwrap()on path conversion withto_string_lossy()to handle non-UTF-8 paths.arg_schema_parse,graph_from_bytes,jq_parse_json,mfile_from_toml) and updates thejust fuzzrecipe to auto-pass a dictionary when a matching.dictfile exists.JqError.Macroscope summarized 617e95e.