CURE: Critical-Token-Guided Re-concatenation for Entropy-collapse Prevention

If you find this project useful, please give us a star 🌟.

• 2024.8.14: We’re excited to announce the release of CURE’s paper.
• 2024.8.22: We’re excited to announce the release of CURE’s model.

In Stage 1, given an input query $q$, the policy model produces a pool of candidate responses. We compute token-level entropy to identify critical tokens (high entropy), extract the clauses immediately preceding those tokens, append them to $q$ to form refined prompts, and query the model again. The newly generated responses are aggregated with the original ones and jointly optimized within a single group. In Stage 2, we continue training to translate the exploration bonus into realized performance.

Comparison of Entropy comparison of CURE-First-Stage and other methods at temperature 1.0.

CURE performs competitively compared with other algorithms. We report avg@32 for AIME24, AIME25, and AMC23 and avg@1 for others.

Our code has been incorporated into VERL as a plugin, located in recipe-CURE

Exactly the same as the environment setup in verl, no additional configuration is required. In our actual workflow, we executed the following operations directly within the released image.

cd CURE
pip install --no-deps -e .
pip install --no-deps git+https://github.com/=hiyouga/MathRuler.git
pip install math_verify

For training, simply download the dataset from DAPO-Math-17K and set its path as TRAIN_FILE in the startup script. We use Qwen-2.5-Math-Base as the training baseline; download it and set its path as CKPTS_DIR in the same script.

We also recommend downloading AIME-2024 and setting its path as TEST_FILE in the same script.

cd CURE
sh ./recipe/CURE_First_Stage/run_cure_stage_1.sh

cd CURE
sh ./recipe/CURE_Second_Stage/run_cure_stage_2.sh

This project has been developed partially based on the following pioneering works on GitHub repositories. We express our profound gratitude for these foundational resources:

• https://github.com/volcengine/verl
• https://github.com/NVlabs/NFT
• https://github.com/huggingface/Math-Verify

We would like to extend our special thanks to the following contributors @Qingbin Li, @Rongkun Xue, for their valuable contributions and support to this algorithm library.

@misc{li2025curecriticaltokenguidedreconcatenationentropycollapse,
      title={CURE: Critical-Token-Guided Re-concatenation for Entropy-collapse Prevention}, 
      author={Qingbin Li and Rongkun Xue and Jie Wang and Ming Zhou and Zhi Li and Xiaofeng Ji and Yongqi Wang and Miao Liu and Zheming Yang and Minghui Qiu and Jing Yang},
      year={2025},
      eprint={2508.11016},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2508.11016}, 
}

All code within this repository is under Apache License 2.0.