[rollout] fix: DeepSeek continuous token builder cannot render tool appends - #7630

What does this PR do?

ContinuousTokenBuilder._tokenize_tool_group() renders a tool group behind a synthetic assistant message whose tool calls carry "arguments": {}. The DeepSeek R1, V3.1 and V3.2 chat templates splice tool['function']['arguments'] into the prompt by string concatenation, so with DeepSeekContinuousTokenBuilder every tool append raises TypeError on that concatenation. DeepSeek-V3 is the only one of the family that gets through, and only because its template renders tool_calls when content is none, which the synthetic message never is.

Found while checking #7628 on the DeepSeek family (#7628 (comment)). The fix is a one-method override: the DeepSeek builder carries the synthetic arguments as the JSON string "{}". The message only exists to be diffed away, so the emitted tokens do not change. #7628 is what makes this sufficient: the history, with the harness's own dict arguments from OpenAIFunctionCallSchema, is no longer re-rendered on a tool append.

The base builder is left alone; Qwen-style templates run arguments through tojson and expect a mapping.

Checklist Before Starting

• Search for similar PRs: https://github.com/verl-project/verl/pulls?q=is%3Apr+deepseek+continuous+token
• PR title formatted as [{modules}] {type}: {description}

Test

• tests/utils/test_continuous_token_on_cpu.py: 124 passed (122 existing + 2 new). The new DeepSeek-shaped fake template concatenates arguments like the real ones; on main the new test fails with the same TypeError.
• Real tokenizers (tokenizer files only, transformers 5.16.1), upstream main @3dab856 vs main with this patch, token ids from tokenize_non_assistant_incremental_messages on tool_agent_loop-shaped trajectories, 24 cases per row (system prompt on/off × 0 / 10 / 50 prior tool turns × 1 or 2 parallel calls × enable_thinking default / False):

With JSON-string arguments the naive full-history suffix diff is available as an independent reference: the fixed output matches it 24/24 on V3.1 and V3.2-Exp. On V3 and R1 it matches on the cases without prior tool turns; those two templates keep a conversation-global is_output_first flag that no bounded render can see, which is pre-existing and unchanged here.

Script and raw output: https://github.com/ruiling-smartbear/verl-experiments/blob/main/generation_prompt_fold_experiment.md#the-fix-checked

API and Usage Example

No API change.

Design & Code Changes

• DeepSeekContinuousTokenBuilder._synthetic_assistant_for_tools: the base message with arguments replaced by "{}".
• Tests: _DeepSeekBoundaryTokenizer, a minimal string-concatenating renderer with no generation prompt after a tool message; one test for the tool append through create_continuous_token_builder(..., model_family="deepseek") (and the base builder raising on the same template), one for the synthetic message itself.

Checklist Before Submitting

• Read the Contribute Guide.
• ruff check / ruff format pass on the touched files.
• Docs: internal tokenizer utility, no user-facing change.
• Unit tests in tests/utils/test_continuous_token_on_cpu.py, covered by the CPU unit-test workflow.
• CI request in ci-request once ready.