ReTool: Reinforcement Learning for Strategic Tool Use in LLMs

• Base model: Qwen/Qwen2.5-32B-Instruct
• SFT dataset: JoeYing/ReTool-SFT
• RL dataset: BytedTsinghua-SIA/DAPO-Math-17k
• Val dataset: yentinglin/aime_2025

1. Data preparation

python3 recipe/retool/retool_sft_preprocess.py

2. Training

bash recipe/retool/run_qwen2-32b_sft.sh

After 6 epoches, validation metrics:

• val-core/aime_2025/acc/mean@30: 0.24
• val-aux/num_turns/mean: 7.2

bash recipe/retool/run_qwen2-32b_dapo.sh

After 150 steps, validation metrics:

• val-core/aime_2025/acc/mean@30: 0.6
• val-aux/num_turns/mean: 10

bash recipe/retool/run_qwen2-32b_ppo.sh

After 250 steps, validation metrics:

• val-core/aime_2025/acc/mean@30: 0.55
• val-aux/num_turns/mean: 8.3