Training-Efficient Text-to-Music Generation with State-Space Modeling

• Training and inference code released (2026/02)
• Data preprocessing pipeline (coming soon)

This project depends on Mamba-SSM and causal-conv-1d. Please install them before setting up the Python environment.

conda create -n ssm_ttm python=3.12
conda activate ssm_ttm
pip install -r requirements.txt

⚠️ Make sure your CUDA / PyTorch versions are compatible with Mamba-SSM.

To train the text-to-music model, run the following command:

python main_pl.py \
  --project_name [experiment_name] \
  --root_path [dataset_root_path] \
  --layer_num [number_of_layers]

• project_name: Name of the experiment (used for logging and checkpoints)
• root_path: Root directory of the preprocessed dataset
• layer_num: Number of SSM-based layers in the model

Please download the pretrained checkpoint and configuration files from Google Drive:

👉 https://drive.google.com/drive/folders/1O3uIUAMx12Y2VsI-Y5zdtwdYCr2PfgiU

You can edit the input text captions at line 77 in pl_inference.py.

python pl_inference.py \
  --model_path [model_ckpt_path] \
  --config_path [model_config_path] \
  --save_dir_name [output_directory_name]

The generated outputs will be saved to:

./outputs/[output_directory_name]

This repository only generates DAC audio tokens. To convert DAC tokens into waveforms, please use the official DAC decoder:

• DAC repository: https://github.com/descriptinc/descript-audio-codec

Make sure the DAC configuration matches the one used during training.

• Currently, only DAC-token-level generation is supported
• Data preprocessing scripts will be released in a future update

If you find this work useful, please consider citing our paper.

@misc{lee2026trainingefficienttexttomusicgenerationstatespace,
      title={Training-Efficient Text-to-Music Generation with State-Space Modeling}, 
      author={Wei-Jaw Lee and Fang-Chih Hsieh and Xuanjun Chen and Fang-Duo Tsai and Yi-Hsuan Yang},
      year={2026},
      eprint={2601.14786},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2601.14786}, 
}