SeTox is a search-augmented framework for Chinese neologism toxicity detection. It combines a tool-use language model with live web search so the model can interpret emerging or implicit terms before making a safety decision.

• Dataset Hub: https://huggingface.co/datasets/thu-coai/SeTox
• Released Model Hubs: https://huggingface.co/thu-coai/SeTox-Qwen2.5-3B and https://huggingface.co/thu-coai/SeTox-Qwen2.5-7B

The repository contains code, examples, training configs, and the release data package. The 3B and 7B model weights are available on Hugging Face. Hugging Face dataset/model card drafts live under hf/, and upload notes are in docs/huggingface_upload.md.

pip install -r requirements.txt

For live search, copy .env.example to .env and set SERPER_API_KEY. Supported search backends are:

• serper: live web search through Serper.
• static: deterministic fixture search with --fixture_path.
• none: disables web search.

For offline smoke tests, use --mock or --search_backend none.

Run a lightweight pipeline check without loading a model:

python scripts/run_inference.py \
  --mock \
  --input_path examples/input.json \
  --output_path outputs/example_predictions.json

python scripts/evaluate.py \
  --input_path outputs/example_predictions.json \
  --output_path outputs/example_metrics.json

Run with a released model and live search:

python scripts/run_inference.py \
  --model_path thu-coai/SeTox-Qwen2.5-3B \
  --search_backend serper \
  --input_path data/eval/neologism_test.json \
  --output_path outputs/neologism_predictions.json

Use thu-coai/SeTox-Qwen2.5-7B for the 7B checkpoint.

Evaluate a prediction file:

python scripts/evaluate.py \
  --input_path outputs/neologism_predictions.json \
  --output_path outputs/neologism_metrics.json

Training uses LLaMA-Factory full SFT. The released training data follows ShareGPT/tool-call formatting and is indexed by data/train/dataset_info.json. Install LLaMA-Factory in the training environment so llamafactory-cli is on PATH, then run:

bash train_code/run.sh train_code/qwen2.5-7b.yaml
bash train_code/run.sh train_code/qwen2.5-3b.yaml

The default configs use public Qwen model ids and local output directories. Adjust model_name_or_path, deepspeed, and output_dir for your environment. The released checkpoint summaries are in results/summary/qwen2.5-3b_text2_neologism.json and results/summary/qwen2.5-7b_text2_neologism.json.

The cleaned reproducibility package includes:

• data/raw/neologism_dct.json: 974 cleaned neologism records.
• data/train/train_tool.json: 995 tool-use training examples.
• data/train/train_normal.json: 600 direct training examples.
• data/eval/neologism_test.json: 624 neologism test examples.
• data/eval/general_test.json: 368 general safety test examples.

The released neologism file preserves the user-provided 974-record source file, including seven duplicated source id values: 79, 132, 569, 591, 373, 427, and 730. We do not silently reindex the raw file; use list order when a stable per-record index is needed. See data/README.md and data/schema.md for details. The tool-use training file is the exact final training snapshot and contains some search-result noise; see docs/data_audit.md for the release-readiness audit.

The code and released data package in this repository are released under the MIT License; see LICENSE.

The released model weights are fine-tuned from Qwen checkpoints and remain subject to the corresponding base model licenses:

• thu-coai/SeTox-Qwen2.5-3B: based on Qwen/Qwen2.5-3B-Instruct, under the Qwen Research License.
• thu-coai/SeTox-Qwen2.5-7B: based on Qwen/Qwen2.5-7B-Instruct, under Apache-2.0.

Before publishing or modifying the release package, run:

python scripts/validate_release.py
python scripts/audit_training_noise.py
python -m compileall setox scripts

The expected main checkpoint metric is within one percentage point of the paper result because live search and generation can introduce minor drift.

If you use SeTox, please cite the ACL paper:

• Paper: https://aclanthology.org/2026.acl-long.1602/
• DOI: https://doi.org/10.18653/v1/2026.acl-long.1602

@inproceedings{cui-etal-2026-setox,
  title = "{S}e{T}ox: Search-enhanced Reasoning for {LLM}-based Toxicity Detection over {C}hinese Internet Buzzwords",
  author = "Cui, Yiming and Zhang, Qinglin and Su, Xu and Min, Changyu and Hu, Shilin and Huang, Minlie",
  editor = "Che, Wanxiang and Nabende, Joyce and Shutova, Ekaterina and Pilehvar, Mohammad Taher",
  booktitle = "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  month = jul,
  year = "2026",
  address = "San Diego, California, USA",
  publisher = "Association for Computational Linguistics",
  url = "https://aclanthology.org/2026.acl-long.1602/",
  doi = "10.18653/v1/2026.acl-long.1602",
  pages = "33018--33034"
}