This is the official implementation of the paper: HyLaT:Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol.

HyLAT enables language model agents to communicate through a hybrid channel that combines continuous latent tokens (compressed cognitive signals) and discrete text tokens (explicit responses). This goes beyond conventional text-only communication by allowing agents to share nuanced, information-dense reasoning states that are too costly to express fully in natural language.

• Train HyLaT
• Evaluation

hylat/
├── hylat/                          # Training code
│   ├── train.py                    # Stage 1 training
│   ├── train_mas_dual.py           # Stage 2 training (2 homogeneous agents)
│   ├── train_mas_general.py        # Stage 2 training (N agents)
│   ├── train_mas_dual_hetero.py    # Stage 2 training (heterogeneous agents)
│   ├── src/
│   │   ├── model.py                # Stage 1 model
│   │   ├── model_decoder_mas_dual.py        # Stage 2 dual-agent model (HyLAT)
│   │   ├── model_decoder_mas_general.py     # Stage 2 N-agent model
│   │   └── model_decoder_mas_dual_hetero.py # Stage 2 heterogeneous model
│   └── configs/
│       ├── stage1_llama.yaml       # Example Stage 1 config
│       ├── stage2_dual_llama.yaml  # Example Stage 2 config
│       ├── stage2_hetero_llama_qwen.yaml  # Example Stage 2 hetero config
│       └── test_mas.yaml           # Evaluation config
└── eval/                           # Evaluation code (MAD tasks)
    ├── scripts/
    │   ├── eval_hylat.sh           # Run HyLaT evaluation
    │   └── eval_nl.sh              # Run NL baseline
    └── src/
        ├── main_latent.py          # Evaluation entry point (HyLaT)
        ├── main.py                 # Evaluation entry point (baselines)
        ├── models/agents/          # Agent implementations
        └── tasks/                  # Task definitions (debate, IA, workflow)

The training framework proceeds in two stages:

Clone the repository:

git clone https://github.com/xymou/hylat.git
cd hylat/hylat

Create environment:

conda create -n hylat python=3.12.12
conda activate hylat
pip install -r requirements.txt

Each training sample is a JSON object with a messages list. Each message contains:

{
  "messages": [
    {
      "role": "user",
      "content": "...",
      "short_answer": "",
      "long_answer": ""
    },
    {
      "role": "assistant",
      "content": "",
      "short_answer": "...",
      "long_answer": "..."
    }  
  ]
}

Each sample is a multi-turn dialogue between two agents (it can also be extended to more than 2 agents):

{
  "messages": [
    {
      "role": "user",
      "question_agent1": "Solve: Janet's ducks lay 16 eggs per day...",
      "question_agent2": "Solve: Janet's ducks lay 16 eggs per day...",
    },
    {
      "role": "assistant",
      "long_answer": "...",
      "short_answer": "...",
      "agent": 1
    },
    {
      "role": "assistant",
      "long_answer": "...",
      "short_answer": "...",
      "agent": 2
    },    
    ...
  ],
  "mask_inter_res": true
}

Stage 1 (single agent):

torchrun --nproc_per_node=8 train.py --conf ./configs/stage1_llama.yaml

Stage 2 (two homogeneous agents):

torchrun --nproc_per_node=8 train_mas_dual.py --conf ./configs/stage2_dual_llama.yaml

Edit the corresponding config file in ./configs/ to set model_name_or_path, data_name, output_dir and etc before running.

We evaluate HyLaT on multi-agent debating (MAD) tasks using a framework built on top of SDE.

Install the evaluation dependencies (separate from training):

cd eval
conda create -n SDE python=3.10.16
conda activate SDE
pip install -r requirements.txt

Configure data and project paths in eval/src/root_path.py:

DATA_ROOT_PATH = "/path/to/data"   # root directory of evaluation datasets
ROOT_PATH      = "/path/to/eval"   # root directory of this eval folder

Prepare the evaluation config

Edit hylat/configs/test_mas.yaml: set ckpt_dir to your trained Stage 2 checkpoint path and ensure model_name_or_path, num_latent, prj_dim match the training config.

Run HyLaT agents on the MAD tasks:

cd eval
bash scripts/eval_hylat.sh

This calls src/main_latent.py with --method latent_stage2 and iterates over all datasets. The key arguments are:

For setting more MAD methods or datasets, plz refer to SDE.

If you find this repo helpful, please cite:

@article{mou2026hylat,
  title   = {HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol},
  author  = {Xinyi Mou and Siyuan Wang and Zejun Li and Yulan He and Zhongyu Wei},
  year    = {2026},
  journal = {arXiv preprint arXiv:2605.25421}
}

@article{shen2025codi,
      title={CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation}, 
      author={Zhenyi Shen and Hanqi Yan and Linhai Zhang and Zhanghao Hu and Yali Du and Yulan He},
      year={2025},
      journal={arXiv preprint arxiv:2502.21074},
}