MELLM: A Flow-Guided Large Language Model for Micro-Expression Understanding

We proposed a novel Micro-expressions Large Language Model (MELLM) for ME Understanding, particularly in interpreting subtle facial dynamics and underlying emotional cues.

• Python == 3.12

conda create -n mellm python=3.12.2
conda activate mellm

git clone https://github.com/zyzhangUstc/MELLM.git
cd MELLM_pipeline

# Install Dependencies
conda install pillow
pip install openface-test
pip install -r requirements.txt

# Download Pretrained Weights
huggingface-cli download --resume-download nutPace/openface_weights --local-dir weights

Modify the file /envs/mellm/lib/python3.12/site-packages/openface/STAR/conf/base.py to fix a hardcoded logging path. In the init_instance(self) function (around line 70), add the following line right after the function definition:

def init_instance(self):
    self.log_dir = os.path.expanduser("~/logs") # new add
    self.writer = SummaryWriter(logdir=self.log_dir, comment=self.type)
...

This ensures logs are written to a valid user directory instead of the original hardcoded path.

download weights from link：https://pan.baidu.com/s/1J35NV76iSPlVjAfHH2NjNA?pwd=s5ku

and

link：https://pan.baidu.com/s/1mNumDG9E9Cf130zfNJCBmw?pwd=jhrj

put them into ckpt fold

python pipeline.py

The training code and dataset will be released as open-source upon acceptance of the paper.

An earlier version of MELLM was based on Qwen-2.5-VL.

Paper: https://arxiv.org/pdf/2505.07007v1

Code: https://github.com/zyzhangUstc/mellmv1-base-on-Qwen2.5vl-7B-

This project relies on code from existing repositories: princeton-vl/WAFT and Qwen3. We thank the original authors for their excellent work.

If you find our project helpful to your research, please consider citing:

 @misc{zhao2025mellmexploringllmpoweredmicroexpression,
      title={MELLM: Exploring LLM-Powered Micro-Expression Understanding Enhanced by Subtle Motion Perception}, 
      author={Sirui Zhao and Zhengye Zhang and Shifeng Liu and Xinglong Mao and Shukang Yin and Chaoyou Fu and Tong Xu and Enhong Chen},
      year={2025},
      eprint={2505.07007},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2505.07007}, 
}