Skip to content

chore(parsers): update the verbatim-arguments release line to 8.1.0 - #4

Open
jamesdborin wants to merge 37 commits into
release/parsers-v1-7.0.1from
upgrade/parsers-v1-8.1.0
Open

jamesdborin wants to merge 37 commits into
release/parsers-v1-7.0.1from
upgrade/parsers-v1-8.1.0

Conversation

@jamesdborin

@jamesdborin jamesdborin commented Sep 21, 2026

Copy link
Copy Markdown

Summary

Update the fork's release/parsers-v1-7.0.1 line to upstream dynamo-parsers 8.1.0 (c4b8f391e83af58781b024894b79620acd323389) for the Dynamo upstream integration. Retain the GLM fixes that preserve XML entities literally, avoid splitting JSON array arguments on commas, and keep string-typed arguments as strings. Keep dynamo-protocols as a registry dependency so git consumers share protocol types with the published crates.

The Dynamo integration branch pins this PR's commit. This targets the existing parser release branch because the fork's main branch is already on parser 9 / protocol 6, whereas the selected Dynamo upstream revision requires parser 8 / protocol 5.

Validation

  • cargo test -p dynamo-parsers --lib: 824 passed, 4 ignored.
  • cargo test -p dynamo-parsers glm47: 31 unit tests and 3 integration tests passed.
  • Canonical conformance HTML rendered successfully from the checked-in fixture snapshot. This is a report render, not a new live capture.
  • Fixed a duplicated token_id test field introduced by the merge.

Review in cubic

keivenchang and others added 30 commits July 30, 2026 23:51
…i-dynamo#167)

Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
…s on one page — NO PARSER CHANGE (ai-dynamo#168)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
…mo#100)

Signed-off-by: Rita Zhang <1856066+ritazh@users.noreply.github.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
…p pin/press — NO PARSER CHANGE (ai-dynamo#169)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
…NO PARSER CHANGE (ai-dynamo#173)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
…olation lane (DIS-2381) (ai-dynamo#135)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
…es — NO PARSER CHANGE (DIS-2408) (ai-dynamo#140)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
… and let vendors supply their own (ai-dynamo#178)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
Signed-off-by: Chanh Nguyen <channguyen@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
release-plz Bot and others added 7 commits August 18, 2026 01:12
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
…treaming (ai-dynamo#190)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: release-plz[bot] <release-plz[bot]@users.noreply.github.com>
Signed-off-by: jamesdborin <james.dborin@gmail.com>
@github-actions github-actions Bot added the chore label Sep 21, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

40 issues found across 171 files

Not reviewed (too large): parsers/v2/src/unified/mod.rs (~3,093 lines), parsers/v2/src/unified/qwen3.rs (~2,634 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="parsers/v1/src/tool_calling/jail/guided_stream.rs">

<violation number="1" location="parsers/v1/src/tool_calling/jail/guided_stream.rs:337">
P1: When a required-call array contains a primitive or nested array before a valid call, this branch leaves the cursor's source index unchanged. The cursor then emits the later call at the wrong index, and completion emits or reconciles the same call at its actual index; advance the source index for every top-level array element, not only object closures.</violation>

<violation number="2" location="parsers/v1/src/tool_calling/jail/guided_stream.rs:414">
P2: When `name` appears before the argument object and is repeated afterward, the cursor sends the first name before it can see the replacement. Reject duplicate top-level `name` keys in the guided grammar or defer commitment until the call object closes.</violation>
</file>

<file name="parsers/v1/Cargo.toml">

<violation number="1" location="parsers/v1/Cargo.toml:7">
P1: The 8.1.0 parser update leaves workspace guided-streaming tests with two incompatible `dynamo-protocols` crates. Align conformance with the parser's registry protocol dependency so `JailedStream` accepts its input type.</violation>
</file>

<file name="parsers/v2/tests/vendor_registry.rs">

<violation number="1" location="parsers/v2/tests/vendor_registry.rs:200">
P2: The parity tests are not synchronized with the registry-mutating tests in this same integration binary, so they can observe a temporary vendor override. Acquire `SERIALIZE` in both parity tests or otherwise isolate these lookups from concurrent registry mutations.</violation>

<violation number="2" location="parsers/v2/tests/vendor_registry.rs:214">
P1: These new parity tests fail in a clean process because unified Qwen3 reports `false` while the tool-only Qwen3-Coder adapter reports `true`. Align the unified parser's decoding requirement with the shared scanner, or update the contract and both expectations consistently.</violation>
</file>

<file name="parsers/v2/src/unified/guided_cursor.rs">

<violation number="1" location="parsers/v2/src/unified/guided_cursor.rs:310">
P1: When a required call has a non-object `arguments`/`parameters` alias followed by a valid object under the other alias, the cursor commits it before validation rejects the element. Mark container values for either alias as non-object, so ambiguous payloads are never put on the wire.</violation>

<violation number="2" location="parsers/v2/src/unified/guided_cursor.rs:338">
P1: When a required-call array contains a scalar or nested-array invalid element before a valid call, the cursor does not advance `self.index` for that element. The later call is streamed with the wrong `tool_index`, then completion cannot match it and emits the same call again; advance the array-element index for every top-level item, not only objects closed by `}`.</violation>
</file>

<file name="conformance/utils/src/_common.sh">

<violation number="1" location="conformance/utils/src/_common.sh:62">
P1: The staged conformance render now fails when importing `tables.markup` because its registry lookup cannot find the registry at `<stage>/src/parser_families.yaml` from the new top-level `tables` location. Copy `parser_families.yaml` beside the staged `tables` package as well.</violation>
</file>

<file name="conformance/utils/src/capture_vllm_rust.py">

<violation number="1" location="conformance/utils/src/capture_vllm_rust.py:49">
P1: When capturing a vLLM 0.23 checkout, these unconditional imports do not exist, so Cargo compilation fails before capturing even supported parsers. Generate version-specific imports and parser arms, or omit these mappings for pre-0.24 layouts.</violation>

<violation number="2" location="conformance/utils/src/capture_vllm_rust.py:219">
P1: When the probe selects the pre-0.25 `tool` dialect, the generated code calls `ToolParserOutput::append`, which is not part of that old field-based output API. Keep dialect-specific append logic and merge `calls`/`normal_text` for the old crate.</violation>

<violation number="3" location="conformance/utils/src/capture_vllm_rust.py:303">
P2: When capturing `gemma4` on vLLM 0.25+, this arm makes native unified-parser cases render as Rust exceptions instead of expected unavailable cells. Propagate an unavailable marker through capture and fixture assembly so these cases render as n/a.</violation>
</file>

<file name="protocols/src/types/chat.rs">

<violation number="1" location="protocols/src/types/chat.rs:732">
P1: When an assistant message contains both `reasoning_content` and `reasoning`, Serde rejects the request as a duplicate field. Accept both keys with a custom deserializer and apply the non-empty `reasoning_content`/`reasoning` fallback.</violation>
</file>

<file name="parsers/v2/src/unified/muse_glimmer.rs">

<violation number="1" location="parsers/v2/src/unified/muse_glimmer.rs:48">
P1: After `initialize_request` selects `Reasoning` or `Response`, `reset` returns the scanner to `Idle` instead of preserving that request state, so the next headerless stream is routed incorrectly. It also leaves `started` set, making the documented initialize-after-reset recovery path fail; preserve the request configuration and clear/reinitialize the scanner lifecycle state during reset.</violation>
</file>

<file name="parsers/v1/src/tool_calling/structural_tag/builder.rs">

<violation number="1" location="parsers/v1/src/tool_calling/structural_tag/builder.rs:218">
P1: When `tool_choice="none"` uses the K2 builder, `build_tool_call_ban()` returns `None`, so the model can still begin a native `<|tool_calls_section_begin|>` tool call. Provide the K2 section marker through the ban-token path.</violation>
</file>

<file name="conformance/fixtures-manifest.json">

<violation number="1" location="conformance/fixtures-manifest.json:5">
P1: The fixture snapshot does not match the parser versions being released. Regenerate and package the Dynamo captures after the bump so the manifest metadata and versioned shards include `8.1.0` and `0.3.2`; otherwise the renderer silently skips the current v1 overlay and uses stale v2 data.</violation>
</file>

<file name="parsers/v1/src/tool_calling/jail/mod.rs">

<violation number="1" location="parsers/v1/src/tool_calling/jail/mod.rs:999">
P1: After the first immediate payload completes, the state is no longer jailed, so a subsequent bare JSON payload bypasses guided streaming and can leak as assistant text. Re-enter immediate jail for each new payload before marker-based handling.</violation>
</file>

<file name="conformance/tests/capture_cross_version.rs">

<violation number="1" location="conformance/tests/capture_cross_version.rs:124">
P2: When `finish()` emits nothing, this capture drops the finish schedule row and no longer matches `unified_render`'s per-chunk contract. Always append the finish row in both native and split capture paths, even when its expected list is empty.</violation>

<violation number="2" location="conformance/tests/capture_cross_version.rs:175">
P2: This capture does not use the same tool definitions as the live harness: it omits `log` and `sum_values`, which occur in the current corpus. Add both schemas so tool availability and argument handling cannot create false cross-version diffs.</violation>
</file>

<file name="conformance/utils/src/resolve_reasoning_fixtures.py">

<violation number="1" location="conformance/utils/src/resolve_reasoning_fixtures.py:99">
P2: When an engine overlay changes only some families, this call stamps every staged reasoning file with the global latest overlay version. Unchanged families still contain anchor or older output, so the table publishes false `captured_with` labels; compute the latest applied overlay per family before stamping.</violation>
</file>

<file name="conformance/utils/src/tables/reasoning/table.py">

<violation number="1" location="conformance/utils/src/tables/reasoning/table.py:487">
P2: When a reasoning candidate contains `error`, this call leaves `err` at `cmp_entry`'s default `0`. The compare view then treats a failed parser as an ordinary output divergence instead of a parser error; pass `err=1` for `err:` signatures.</violation>
</file>

<file name="conformance/utils/src/refresh_dynamo_captures.py">

<violation number="1" location="conformance/utils/src/refresh_dynamo_captures.py:338">
P2: When `--label` names an existing capture, `refresh_stream` deletes that directory before recording the new output. Reject an existing labeled directory before refreshing, or change the option’s contract so this destructive replacement is explicit.</violation>
</file>

<file name="conformance/utils/piece1_coupling_gate.sh">

<violation number="1" location="conformance/utils/piece1_coupling_gate.sh:52">
P2: When `gen_unified_golden.py` is missing, this guard silently skips the scenario-count check and can print PASS. Add a failure branch for the missing required generator.</violation>

<violation number="2" location="conformance/utils/piece1_coupling_gate.sh:66">
P2: This check rejects the repository's generator with `81, expected 33`, so the gate cannot pass for the checked-in corpus. Compare against the selected `BASE` generator count or update the expected shape instead of hard-coding 33.</violation>

<violation number="3" location="conformance/utils/piece1_coupling_gate.sh:73">
P2: When `parsers/v2/src/unified/mod.rs` is missing, this guard silently skips every required peer-surface check and can report PASS. Add a failure branch for the missing required module.</violation>
</file>

<file name="parsers/v2/src/tool_calling/muse_glimmer.rs">

<violation number="1" location="parsers/v2/src/tool_calling/muse_glimmer.rs:517">
P2: When an orphan marker removal respells another marker across two emitted runs, this per-run stripping leaks the respelled control token and makes output depend on chunking. Carry a splice-aware suffix across text/reasoning runs, or delay emission until marker removal cannot create a new marker.</violation>

<violation number="2" location="parsers/v2/src/tool_calling/muse_glimmer.rs:736">
P2: After resetting an initialized Muse parser, the next stream loses its prompt-prefilled channel and cannot be reinitialized. Preserve the configured starting state and reset the lifecycle flag (or separate reset state from finish state) so retry parsing keeps the request configuration.</violation>
</file>

<file name="conformance/utils/src/parser_families.yaml">

<violation number="1" location="conformance/utils/src/parser_families.yaml:38">
P2: Because Muse is not present in a released vLLM, this non-null mapping makes the capture harness try to instantiate an unregistered parser instead of recording vLLM as unavailable. Set `vllm_python` to `null` until a release carries the parser.</violation>
</file>

<file name="conformance/utils/src/markers.py">

<violation number="1" location="conformance/utils/src/markers.py:222">
P2: For an `exception` block, the structured comparison facts still look like a missing parser: `present` is false and `error_kind` is null. Extend the facts/error classification (and the comparison output path where needed) so a parser that ran and raised remains distinguishable from `unavailable` throughout the model.</violation>
</file>

<file name="conformance/utils/src/dynamo_version.py">

<violation number="1" location="conformance/utils/src/dynamo_version.py:34">
P2: When a change-scoped label such as `0.1.24+pr163` is used, the stream resolver treats it as the same version as `0.1.24` and folds both captures into the default release snapshot. Exclude tagged captures from the default fold while retaining them as explicitly selectable historical candidates.</violation>
</file>

<file name="conformance/utils/src/capture_peer_versions.py">

<violation number="1" location="conformance/utils/src/capture_peer_versions.py:734">
P2: When capturing a pre-0.25 vLLM source checkout, this unconditional `gemma4` branch discards real parser output and records it as unavailable. Restrict the unavailable marker to the unified-parser dialect; preserve `_rust_captured_case_doc(cap)` for the legacy tool-parser dialect.</violation>
</file>

<file name="parsers/v1/src/tool_calling/structural_tag/kimi_k2.rs">

<violation number="1" location="parsers/v1/src/tool_calling/structural_tag/kimi_k2.rs:95">
P2: Kimi K2 auto decoding still permits `<|tool_call_begin|>` outside the tool-call section, and the parser then recovers it as a valid call. Add xgrammar’s reserved-marker exclusions to the triggered format.</violation>
</file>

<file name="conformance/utils/README.md">

<violation number="1" location="conformance/utils/README.md:26">
P2: When an existing parser cannot be captured, the harness also records `unavailable` with a `parser not captured` reason, which renders as `✗` rather than grey `n/a`. Describe this field as including not-captured cases and distinguish those parser errors from benign unavailable cases.</violation>
</file>

<file name="conformance/tests/unified_render.rs">

<violation number="1" location="conformance/tests/unified_render.rs:664">
P2: When the live capture differs from the authored expectation or is missing, the vLLM cell and legend report different results because `vllm_red` is counted before the live lookup. Count the legend from the live classification and update the standalone header/feed metadata to describe the live vLLM Rust column.</violation>
</file>

<file name="conformance/fixtures/unified/README.md">

<violation number="1" location="conformance/fixtures/unified/README.md:29">
P2: The documented release-aware label and test do not exist, so 0.1.22/0.1.23 will still render as `Combined & Unified` and misdescribe empty split-only results. Add the release-aware renderer and regression test, or remove this claimed behavior from the README.</violation>
</file>

<file name="conformance/README.md">

<violation number="1" location="conformance/README.md:115">
P2: The blanket append-only rule is false for batch-on-stream captures. State that only batch, stream, and reasoning peer overlays create version directories; batch-on-stream is refreshed in place.</violation>

<violation number="2" location="conformance/README.md:123">
P2: The vLLM Rust cell points to a script that does not capture Rust and preserves its existing block. Mark it as preserved and direct authors to `capture.sh batch-on-stream` for the Rust refresh.</violation>
</file>

<file name="conformance/utils/src/generate_conformance_table.py">

<violation number="1" location="conformance/utils/src/generate_conformance_table.py:355">
P2: When a peer parser raises an exception, `cell_for` emits `✗`, but `_compute_stats` does not classify `✗` as an error and counts it as documented. Add an explicit `✗` branch to the aggregate error counter so the overview reports parser failures correctly.</violation>
</file>

<file name="conformance/utils/src/assets/conformance.js">

<violation number="1" location="conformance/utils/src/assets/conformance.js:212">
P2: When the selected Compare parser throws and no other Compare produces output, red-on-diff cells display `=` even though no comparison supports agreement. Count `err` entries as no-data when choosing the marker.</violation>
</file>

<file name="conformance/utils/src/pyproject.stub.toml">

<violation number="1" location="conformance/utils/src/pyproject.stub.toml:6">
P2: Until a `vllm_python-0.26.0` reasoning overlay is packaged, this pin leaves the reasoning tab on vLLM 0.24.0 while the tool-calling tabs use 0.26.0. Add and package the 0.26.0 reasoning capture before changing the pin, or keep the pin at a version captured for every corpus.</violation>
</file>

<file name="conformance/utils/tests/test_unified_vllm_expectation.py">

<violation number="1" location="conformance/utils/tests/test_unified_vllm_expectation.py:13">
P2: The documented packaged-tab guard does not exist, so regressions in the `CONFORMANCE_v2.html` missing-capture rendering can ship without a test. Add the packaged-tab assertion or remove this claim and provide equivalent coverage elsewhere.</violation>
</file>

Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic

':' if self.depth == self.key_depth && self.slot == Slot::Colon => {
self.slot = Slot::Value;
}
',' if self.depth == self.key_depth => {

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: When a required-call array contains a primitive or nested array before a valid call, this branch leaves the cursor's source index unchanged. The cursor then emits the later call at the wrong index, and completion emits or reconciles the same call at its actual index; advance the source index for every top-level array element, not only object closures.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At parsers/v1/src/tool_calling/jail/guided_stream.rs, line 337:

<comment>When a required-call array contains a primitive or nested array before a valid call, this branch leaves the cursor's source index unchanged. The cursor then emits the later call at the wrong index, and completion emits or reconciles the same call at its actual index; advance the source index for every top-level array element, not only object closures.</comment>

<file context>
@@ -0,0 +1,962 @@
+            ':' if self.depth == self.key_depth && self.slot == Slot::Colon => {
+                self.slot = Slot::Value;
+            }
+            ',' if self.depth == self.key_depth => {
+                self.slot = Slot::Key;
+                self.pending_key = None;
</file context>
Fix with cubic

Comment thread parsers/v1/Cargo.toml
name = "dynamo-parsers"
readme = "README.md"
version = "7.0.1"
version = "8.1.0"

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The 8.1.0 parser update leaves workspace guided-streaming tests with two incompatible dynamo-protocols crates. Align conformance with the parser's registry protocol dependency so JailedStream accepts its input type.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At parsers/v1/Cargo.toml, line 7:

<comment>The 8.1.0 parser update leaves workspace guided-streaming tests with two incompatible `dynamo-protocols` crates. Align conformance with the parser's registry protocol dependency so `JailedStream` accepts its input type.</comment>

<file context>
@@ -4,7 +4,7 @@
 name = "dynamo-parsers"
 readme = "README.md"
-version = "7.0.1"
+version = "8.1.0"
 edition.workspace = true
 description = "Reasoning and tool-calling parsers for OpenAI-compatible inference output."
</file context>
Fix with cubic

fn canonical_family_matches_the_tool_only_adapter() {
let unified = create_unified_parser_for_family("qwen3", &[]).expect("qwen3 unified");
assert_eq!(
unified.preserve_special_tokens(),

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: These new parity tests fail in a clean process because unified Qwen3 reports false while the tool-only Qwen3-Coder adapter reports true. Align the unified parser's decoding requirement with the shared scanner, or update the contract and both expectations consistently.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At parsers/v2/tests/vendor_registry.rs, line 214:

<comment>These new parity tests fail in a clean process because unified Qwen3 reports `false` while the tool-only Qwen3-Coder adapter reports `true`. Align the unified parser's decoding requirement with the shared scanner, or update the contract and both expectations consistently.</comment>

<file context>
@@ -0,0 +1,230 @@
+    fn canonical_family_matches_the_tool_only_adapter() {
+        let unified = create_unified_parser_for_family("qwen3", &[]).expect("qwen3 unified");
+        assert_eq!(
+            unified.preserve_special_tokens(),
+            tool_only_value(),
+            "unified and tool-only must not report different decoding requirements"
</file context>
Fix with cubic

'}' | ']' => {
let closing_args =
self.element.in_args && self.depth == self.key_depth + 1 && ch == '}';
let closes_element = self.depth == self.key_depth && ch == '}';

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: When a required-call array contains a scalar or nested-array invalid element before a valid call, the cursor does not advance self.index for that element. The later call is streamed with the wrong tool_index, then completion cannot match it and emits the same call again; advance the array-element index for every top-level item, not only objects closed by }.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At parsers/v2/src/unified/guided_cursor.rs, line 338:

<comment>When a required-call array contains a scalar or nested-array invalid element before a valid call, the cursor does not advance `self.index` for that element. The later call is streamed with the wrong `tool_index`, then completion cannot match it and emits the same call again; advance the array-element index for every top-level item, not only objects closed by `}`.</comment>

<file context>
@@ -0,0 +1,1041 @@
+            '}' | ']' => {
+                let closing_args =
+                    self.element.in_args && self.depth == self.key_depth + 1 && ch == '}';
+                let closes_element = self.depth == self.key_depth && ch == '}';
+                self.depth -= 1;
+                if closing_args {
</file context>
Fix with cubic

# way here as it does from conformance/utils/src in the repo — one import name, both
# contexts. The tests/parity/ subtree below stays the fixture + template layout the
# generator's path constants and relative-link resolution are written against.
\cp -Rf "$TOOLS/tables" "$STAGE/tables"

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The staged conformance render now fails when importing tables.markup because its registry lookup cannot find the registry at <stage>/src/parser_families.yaml from the new top-level tables location. Copy parser_families.yaml beside the staged tables package as well.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At conformance/utils/src/_common.sh, line 62:

<comment>The staged conformance render now fails when importing `tables.markup` because its registry lookup cannot find the registry at `<stage>/src/parser_families.yaml` from the new top-level `tables` location. Copy `parser_families.yaml` beside the staged `tables` package as well.</comment>

<file context>
@@ -53,35 +53,39 @@ CARGO="${CARGO:-cargo}"
+  # way here as it does from conformance/utils/src in the repo — one import name, both
+  # contexts. The tests/parity/ subtree below stays the fixture + template layout the
+  # generator's path constants and relative-link resolution are written against.
+  \cp -Rf "$TOOLS/tables" "$STAGE/tables"
   # Family marker declarations (DIS-2442): the staged tests/parity/markup.py resolves
   # its registry at <stage-root>/src/parser_families.yaml, mirroring the repo layout.
</file context>
Suggested change
\cp -Rf "$TOOLS/tables" "$STAGE/tables"
\cp -Rf "$TOOLS/tables" "$STAGE/tables"
\cp -f "$TOOLS/parser_families.yaml" "$STAGE/tables/parser_families.yaml"
Fix with cubic

}
Slot::Value => {
if self.pending_key.as_deref() == Some("name") {
self.element.name = Some(literal);

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When name appears before the argument object and is repeated afterward, the cursor sends the first name before it can see the replacement. Reject duplicate top-level name keys in the guided grammar or defer commitment until the call object closes.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At parsers/v1/src/tool_calling/jail/guided_stream.rs, line 414:

<comment>When `name` appears before the argument object and is repeated afterward, the cursor sends the first name before it can see the replacement. Reject duplicate top-level `name` keys in the guided grammar or defer commitment until the call object closes.</comment>

<file context>
@@ -0,0 +1,962 @@
+            }
+            Slot::Value => {
+                if self.pending_key.as_deref() == Some("name") {
+                    self.element.name = Some(literal);
+                }
+            }
</file context>
Fix with cubic

peers = [
"sglang[diffusion]==0.5.14",
"vllm[flashinfer,runai,otel]==0.24.0",
"vllm[flashinfer,runai,otel]==0.26.0",

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Until a vllm_python-0.26.0 reasoning overlay is packaged, this pin leaves the reasoning tab on vLLM 0.24.0 while the tool-calling tabs use 0.26.0. Add and package the 0.26.0 reasoning capture before changing the pin, or keep the pin at a version captured for every corpus.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At conformance/utils/src/pyproject.stub.toml, line 6:

<comment>Until a `vllm_python-0.26.0` reasoning overlay is packaged, this pin leaves the reasoning tab on vLLM 0.24.0 while the tool-calling tabs use 0.26.0. Add and package the 0.26.0 reasoning capture before changing the pin, or keep the pin at a version captured for every corpus.</comment>

<file context>
@@ -3,5 +3,5 @@
 peers = [
   "sglang[diffusion]==0.5.14",
-  "vllm[flashinfer,runai,otel]==0.24.0",
+  "vllm[flashinfer,runai,otel]==0.26.0",
 ]
</file context>
Fix with cubic

PYEOF
cnt=$( (cd conformance/utils && PYTHONPATH=src python3 "$_cnt_py") 2>/dev/null )
rm -f "$_cnt_py"
if [ "$cnt" = "33" ]; then note "generator scenarios" "33 (99 cases) — main's shape"

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This check rejects the repository's generator with 81, expected 33, so the gate cannot pass for the checked-in corpus. Compare against the selected BASE generator count or update the expected shape instead of hard-coding 33.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At conformance/utils/piece1_coupling_gate.sh, line 66:

<comment>This check rejects the repository's generator with `81, expected 33`, so the gate cannot pass for the checked-in corpus. Compare against the selected `BASE` generator count or update the expected shape instead of hard-coding 33.</comment>

<file context>
@@ -0,0 +1,86 @@
+PYEOF
+  cnt=$( (cd conformance/utils && PYTHONPATH=src python3 "$_cnt_py") 2>/dev/null )
+  rm -f "$_cnt_py"
+  if [ "$cnt" = "33" ]; then note "generator scenarios" "33 (99 cases) — main's shape"
+  elif [ -z "$cnt" ]; then bad "generator scenarios" "count FAILED to run — check manually"
+  else bad "generator scenarios" "$cnt, expected 33 — corpus expansion belongs to piece 2"; fi
</file context>
Fix with cubic

# parser (`vllm_parser::unified::Gemma4UnifiedParser`) on a tokenizer-backed API
# that this probe cannot drive. Recorded as unavailable, not as a failure.
"gemma4_arm": (
'"gemma4" => anyhow::bail!(\n'

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When capturing gemma4 on vLLM 0.25+, this arm makes native unified-parser cases render as Rust exceptions instead of expected unavailable cells. Propagate an unavailable marker through capture and fixture assembly so these cases render as n/a.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At conformance/utils/src/capture_vllm_rust.py, line 303:

<comment>When capturing `gemma4` on vLLM 0.25+, this arm makes native unified-parser cases render as Rust exceptions instead of expected unavailable cells. Propagate an unavailable marker through capture and fixture assembly so these cases render as n/a.</comment>

<file context>
@@ -270,14 +283,53 @@
+        # parser (`vllm_parser::unified::Gemma4UnifiedParser`) on a tokenizer-backed API
+        # that this probe cannot drive. Recorded as unavailable, not as a failure.
+        "gemma4_arm": (
+            '"gemma4" => anyhow::bail!(\n'
+            '            "gemma4 moved to the native unified parser in vLLM 0.25.0; \\\n'
+            '             not exposed via the tool::ToolParser probe"\n'
</file context>
Fix with cubic


This file guards the AUTHORED spec only. `expect.*` never reaches the packaged shards,
so the published `CONFORMANCE_v2.html` Unified tab cannot read this note at all; it
derives its own from the missing capture. `test_unified_tab_vllm_caveat.py` guards that

@cubic-dev-ai cubic-dev-ai Bot Sep 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The documented packaged-tab guard does not exist, so regressions in the CONFORMANCE_v2.html missing-capture rendering can ship without a test. Add the packaged-tab assertion or remove this claim and provide equivalent coverage elsewhere.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At conformance/utils/tests/test_unified_vllm_expectation.py, line 13:

<comment>The documented packaged-tab guard does not exist, so regressions in the `CONFORMANCE_v2.html` missing-capture rendering can ship without a test. Add the packaged-tab assertion or remove this claim and provide equivalent coverage elsewhere.</comment>

<file context>
@@ -0,0 +1,68 @@
+
+This file guards the AUTHORED spec only. `expect.*` never reaches the packaged shards,
+so the published `CONFORMANCE_v2.html` Unified tab cannot read this note at all; it
+derives its own from the missing capture. `test_unified_tab_vllm_caveat.py` guards that
+side, and neither guard implies the other.
+
</file context>
Fix with cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants