A high-performance key-value merging utility with modular design and automated testing.

• ​ZSMerge Core Module: Efficient key-value cache merging algorithms
• ​Extensible Testing: Comprehensive unittest coverage
• ​Script Automation: Bash scripts for experiment replication
• ​MIT Licensed: Open and permissive usage

• Python 3.8+
• Bash shell (Linux/macOS/WSL)

pip install -r requirements.txt

Create a .env file in the root directory of the project. Example content:

ACCESS_TOKEN=your_huggingface_key_here  

python -m unittest tests.test_mistral_att
python -m unittest tests.test_llama_att
python -m unittest tests.test_llama3_att
python -m unittest tests.test_falcon_att
python -m unittest tests.test_qwen2_att

bash scripts/e_throughput.sh

bash scripts/e_rouge.sh

This project is licensed under the MIT License - see the LICENSE file for details.

If you found this repository helpful in your work, please cite our paper:

@article{liu2025zeromerge,
  title={ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs},
  author={Liu, Xin and Liu, Pei and Tang, Guoming},
  journal={arXiv preprint arXiv:2503.10714},
  year={2025}
}