Official implementation for "Multimodal Chain-of-Thought Reasoning in Language Models" (stay tuned and more will be updated)

Simple RL training for reasoning

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

GLM (General Language Model)

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

名校公开课程评价网

⚡FlashRAG: A Python Toolkit for Efficient RAG Research (WWW2025 Resource)

Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo

[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

Chinese-LLaMA 1&2、Chinese-Falcon 基础模型；ChatFlow中文对话模型；中文OpenLLaMA模型；NLP预训练/指令微调数据集

[NeurIPS'24] HippoRAG is a novel RAG framework inspired by human long-term memory that enables LLMs to continuously integrate knowledge across external documents. RAG + Knowledge Graphs + Personali…

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

Reference implementation for DPO (Direct Preference Optimization)

A PyTorch Library for Multi-Task Learning

The full minitorch student suite.

推荐/广告/搜索领域工业界经典以及最前沿论文集合。A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.

Efficient Retrieval Augmentation and Generation Framework

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Multimodal-GPT

这个仓库有1426个star，不信你试试

A curated list of Multimodal Related Research.

ReCall: Learning to Reason with Tool Call for LLMs via Reinforcement Learning

Xwin-LM: Powerful, Stable, and Reproducible LLM Alignment

[NeurIPS 2024] SimPO: Simple Preference Optimization with a Reference-Free Reward

Official implementation of paper "MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens"

[NIPS2023] RRHF & Wombat