[tool] feat: add Gemma4 tool parser with stop token and response formatting - #6406

What does this PR do?

Adds Gemma4 support to the multi-turn agent loop for tool calling. Gemma4 has two differences from existing models (Qwen, GPT-OSS) that prevent tool calling from working:

1. No EOS after tool calls — Gemma4 generates <|tool_call>call:func{args}<tool_call|> then immediately continues generating. Without an explicit stop token, vLLM never stops and the model generates all tool
   calls in one turn, hallucinating responses.

2. Tool response format requires function name — Gemma4's chat template needs <|tool_response>response:FUNC_NAME{value:"content"}<tool_response|>. The standard apply_chat_template({"role": "tool"}) path
   silently drops the content because it lacks the function name context.

Changes

verl/experimental/agent_loop/tool_parser.py

• Add stop_token_ids property to ToolParser base class (returns [] by default)
• Add Gemma4ToolParser class with stop_token_ids=[49] (<tool_call|> token)
• Parse Gemma4's tool call format: <|tool_call>call:func{args}<tool_call|>

verl/experimental/agent_loop/tool_agent_loop.py

• Inject stop_token_ids from tool parser into sampling_params before generation
• Add Gemma4 special case for tool response formatting (same pattern as existing gpt-oss case)

Config

actor_rollout_ref:
  rollout:
    multi_turn:
      format: gemma4  # Selects Gemma4ToolParser

Tested with Gemma4-26B-A4B on multi-turn tool dataset

Checklist Before Starting

• Search for similar PRs. Paste at least one query link here: https://github.com/verl-project/verl/pulls?q=is%3Apr+gemma4+tool+parser
• Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
  • {modules} include fsdp, megatron, veomni, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward, fully_async, one_step_off
  • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
  • {type} is in feat, fix, refactor, chore, test
  • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
  • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

No stop token, no response fix - 0%
Stop token only - 13.6%
Stop token + response formatting - 59.2%

Design & Code Changes

Follows the same pattern as the existing gpt-oss tool parser implementation:

• Gemma4ToolParser in tool_parser.py — parses Gemma4's special-token tool call format
• Response injection in tool_agent_loop.py — manual formatting when apply_chat_template can't handle the model's response format

One addition to the base class: stop_token_ids property on ToolParser (returns [] by default). This is needed because Gemma4 doesn't emit EOS after tool calls — unlike Qwen/GPT-OSS which do. The agent loop
injects these into sampling_params before generation.

Checklist Before Submitting

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: Not feasible to add CI test — requires Gemma4 model weights and multi-turn environment. Tested end-to-end on 8×H200 with full training pipeline (FSDP2 + vLLM async rollout).
• Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace. (If not accessible, please try the Feishu group (飞书群).)
• If your PR is related to the recipe submodule, please also update the reference to the submodule commit via git submodule update --remote or cd recipe && git pull origin main.