English | 中文

Open-source companion repository for the technical report SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD.

• Model: SLAIAITP/DeepSeek-V4-Flash-OR
• Code: SLAI-AITP/SLAI-T-Rex
• Training image: quay.io/slai-t-rex/slai-t-rex:v1.0.0-a3-cann9.1.0

The report studies full-parameter post-training of DeepSeek-V4 on Ascend CloudMatrix384 SuperPOD (910C). This repository ships the reproducible pieces: OR CPT/SFT data construction, MindSpeed-LLM launch templates, checkpoint conversion, and OR benchmark evaluation.

Two connected goals:

• System scaling: trillion-parameter-class MoE training on Ascend SuperPOD (parallelism, communication, CPU–NPU coordination, AscendC kernels).
• Domain specialization: adapt DeepSeek-V4-Flash to Operations Research with solver-grounded CPT, self-distilled SFT, and benchmark evaluation.

• 34.22% MFU on DeepSeek-V4-Pro, 2.93x vs the open baseline recipe on Ascend SuperPOD.
• AuraKernel: AscendC optimizations for sparse attention, RMSNorm, lightning indexer gradients, RoPE, limited SwiGLU, and mHC-related chains.
• OR CPT–SFT for DeepSeek-V4-Flash: collected OR resources, solver-verified synthetic documents, self-distilled SFT, Clean-CoT gates.
• 10K high-quality SFT samples (four OR task categories, three problem representations).
• 71.81% average zero-shot Pass@1 on NL4OPT / OptiBench / B4O-Feasible / B4O-ORGEval (+3.98 vs GPT-5.4-Mini, +11.27 vs base DeepSeek-V4-Flash in the report).
• CPT→SFT transfer (same SFT): B4O-Feasible 71.22%, B4O-ORGEval 59.39%.

SLAI-T-Rex/
├── data/cpt/            OR-CPT engine (solver-verified synthesis)
├── data/sft/            OR SFT distillation toolkit
├── training/convert/    data and checkpoint conversion
├── training/common/     shared MindSpeed runtime discovery
├── training/cpt/        DeepSeek-V4 Flash/Pro 4K CPT launchers
├── training/sft/        DeepSeek-V4 Flash 8K and Pro 4K SFT launchers
├── kernel/               AscendC kernels, framework patches, and runtime setup
├── docker/               Ascend A3/CANN image build and validation
├── eval/                OR benchmarks
├── docs/                technical report PDF
└── assets/

checkpoint prep -> CPT data -> CPT train -> SFT data -> SFT train -> eval

Large-scale production inputs, private cluster configs, and proprietary eval artifacts are not included. Training scripts assume an existing Ascend / MindSpeed environment (MindSpeed-LLM, CANN, and cluster launchers are not vendored).

The Ascend A3 training image with CANN 9.1.0 is available from Quay.io:

docker pull quay.io/slai-t-rex/slai-t-rex:v1.0.0-a3-cann9.1.0

See the training image build guide for the pinned components and instructions for rebuilding or customizing the image.

SFT data dry-run (no LLM):

git clone https://github.com/SLAI-AITP/SLAI-T-Rex.git
cd SLAI-T-Rex/data/sft
python3 -m pip install -e .
python3 -m or_data_distill validate-sft --input seeds/public_seed.jsonl
python3 -m or_data_distill run --config examples/configs/demo.yaml --dry-run

Real SFT generation: copy configs/run.example.yaml, set llm.base_url / model, then python3 -m or_data_distill run --config configs/run.local.yaml.

CPT engine smoke (needs gurobipy + an OpenAI-compatible endpoint for LLM stages):

cd SLAI-T-Rex/data/cpt
python3 -m pip install -e .
python3 -m or_cpt_engine.cli.main --help

Training, conversion, and eval commands live in the module READMEs above.

Training paths are provided through environment variables. The launchers discover MindSpeed repositories cloned next to SLAI-T-Rex or use explicit MINDSPEED_LLM_DIR, MINDSPEED_DIR, and MEGATRON_DIR values. Match hardware layout, checkpoint format, tokenizer, and packing before changing the defaults.

@misc{li2026slaitrexfullparameterposttraining,
      title={SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD}, 
      author={Dongfang Li and Xiaodong Luo and Ruoyu Sun and Xuhui Chen and Linyuan Qiu and Jian Meng and Zhengxuan Lu and Yiting Wang and Yucheng Xie and Tao Guo and Tianxiang Fang and Jing Li and Sihang Chen and Shihao Hong and Chang Liu and Weihua Dai and Zirong Zeng and Ziwei Zhu and Zhuohan Wang and Zhengjun Yue and Igor Vasilyev and Min Liu and Weijian Sun and Xin Chen and Yingmeng Gao and Jinhua Zhou and Taolue Chen and Chenwei Wu and Dong Zhang and Wenlong Jin and Jinmin Xiang and Barkova Maria and Ushakov Anton and Xianfei Jin and Tian Ding and Zhihang Lin and Qian Chen and Linxin Yang and Mingzhe Yang and Bingwei Zhang and Hongzhang Yang and Fangxue Zhang and Shijun Qin and Jie Yu and Cuihua Hu and Tolstykh Vasiliy and Nosov Ivan and Abdullin Amir and Zhicheng Zhou and Xin Zhang and Zhixiong Ning and Xutong Zhao and Junjie Huang and Jiajun Liu and Weiyan Kong and Zheng Zhang and Wenhan Luo and Lin Hu and Yangbo Guo and Li Zeng and Shihao Zhang and Baotian Hu and Min Zhang and Haizhou Li and Zhiquan Luo},
      year={2026},
      eprint={2607.20145},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2607.20145}, 
}