This repository systematically tracks the structural transition from Modular Assembly — late-fusion / grafted compositions that suffer from a fundamental blindness to raw sensory signals — to Native Multimodal Modeling (NMM), where multiple modalities are intrinsically integrated into a unified transformer space or joint backbone.

⭐ Star this repo to track the latest landmark works. PRs are warmly welcomed for any model we may have missed.

We formalize the NMM ecosystem through a dual-dimensional lens based on Integration Depth (mid-fusion vs. early-fusion) and Functional Input–Output Duality:

Native scaling frameworks that ground cross-modal inputs into linguistic streams for logical reasoning.

Modularly assembled via shallow projectors; blind to raw sensory signals.

• LLaVA [Liu et al., 2023] — 💻 GitHub · 📄 Paper
• DeepSeek-VL [Lu et al., 2024] — 💻 GitHub · 📄 Paper
• Qwen-Image [Wu et al., 2025] — 💻 GitHub · 🌐 Blog

Foundational pioneers maintaining explicit, modality-aware boundaries.

• CogVLM [Wang et al., 2023] — 💻 GitHub
• Qwen-Audio [Chu et al., 2023] — 💻 GitHub · 🌐 Project Page

Massive state-of-the-art evolved mid-fusion architectures:

• Qwen2.5-VL [Qwen Team, 2025] — 💻 GitHub · 🌐 Blog
• Qwen3-VL [Qwen Team, 2025] — 💻 GitHub · 📄 Paper
• InternVL-3.5 [Chen et al., 2025] — 💻 GitHub · 🤗 HF Collection

Scale-driven industrial mid-fusion implementations:

• GLM-4.5V / GLM-V [ZhipuAI, 2025–2026] — 💻 GitHub · 🤗 HF Model
• Kimi K2 / K2.5 [Moonshot AI, 2025–2026] — 🌐 Project Page · 💻 GitHub Org

• MiniCPM-V 4.x [Yu et al., 2025] — 💻 GitHub
• Nemotron 3 Nano Omni [NVIDIA, 2026] — 💻 GitHub · 📄 Paper
• MiMo-V2.5 [Xiaomi MiMo Team, 2026] — 💻 GitHub · 🌐 Project Page
• Gemma-4 / Qwen3.6 — Timeline benchmarks driving advanced contextual reasoning (forthcoming).

Bypassing traditional post-hoc decoders to synthesize photorealistic spatiotemporal physics or continuous speech directly.

• Wan 2.2-T2V-A14B [Wan Team, 2025] — 🤗 HF Model — Unifies video patches into native generation spaces with continuous physics.
• HunyuanVideo & HunyuanVideo-1.5 [Tencent, 2024–2025] — 💻 GitHub · 🤗 HF Model (1.5)
• Kling-Omni [Kuaishou, 2025] — 🌐 Project Page

• OmniVoice [Zhu et al., 2026] — 💻 GitHub · 🌐 Project Page
• MiniCPM-o 2.6 / 4.5 [OpenBMB, 2025–2026] — 🤗 HF Model · 💻 GitHub
• Seedream 3.0 [Gao et al., 2025] — 📄 Tech Report · 🌐 Project Page
• HiDream-I1 — 💻 GitHub

• LTX-2 / LTX-Video [Lightricks, 2024–2026] — 💻 GitHub
• Ming-Flash-Omni [Ant Group / inclusionAI, 2025] — 💻 GitHub · 📄 Paper

Omni-directional unified spaces establishing a symmetric paradigm where comprehension and generation natively coexist.

Born-native designs treating all modalities equivalently via one unified backbone & embedding space.

• Transfusion [Zhou et al., 2024] — 📄 Paper
• Chameleon ★ [Meta AI, 2024] — 💻 GitHub · 📄 Paper
• AnyGPT ★ [Zhan et al., 2024] — 💻 GitHub · 📄 Paper

• Moshi ★ [Défossez et al., 2024] — 💻 GitHub · 📄 Paper — Real-time conversational audio-text dual-stream processing.
• Emu3 / Emu3.5 ★ [BAAI, 2024–2025] — 🌐 Project Page · 📄 Paper — Next-token sequence prediction unifying understanding and synthesis.

• BAGEL-7B [ByteDance Seed Team, 2025] — 🤗 HF Model · 🌐 Project Page · 📄 Paper
• OneCAT-3B [Meituan & SJTU, 2025] — 💻 GitHub · 🤗 HF Model
• Show-o2-7B [Xie et al., 2025] — 💻 GitHub · 📄 Paper

Collapsing representation boundaries.

• Janus-Pro ★ [DeepSeek-AI, 2025] — 🤗 HF Model · 📄 Paper
• Llama-4 Scout / Maverick [Meta AI, 2025] — 🌐 Llama Site — Advanced interleaved-scale exploration.
• LLaDA-V [Ml-GSAI, 2025] — 💻 GitHub · 🌐 Project Page · 📄 Paper
• Lance [ByteDance, 2026] — 📄 Paper — Leading edge of complete native convergence.
• TUNA-2 [Liu et al., 2026] / Mamoda 2.5 [Shi et al., 2026] / LongCat-Next — Forthcoming.
• Dynin-Omni [Kim et al., 2026] — 💻 GitHub · 🤗 HF Model · 📄 Paper · 🌐 Project Page

★ Denotes early exploratory or foundational dual-regime architectures.

Following the systemic structure detailed across Sections §3–§7 of the roadmap paper, the core components of the NMM lifecycle are curated below.

Contributions are very welcome! If a notable native multimodal model is missing or you find an outdated link, please open an Issue or send a Pull Request.

The preferred entry format is:

- **<Model Name>** [<Authors / Team>, <Year>] — [`💻 GitHub`](https://...) · [`📄 Paper`](https://...)

If our formalization, taxonomy, or roadmap framework assists your research, please cite our definitive paper:

@article{TencentYoutuLab2026toward,
  title   = {Toward Native Multimodal Modeling: A Roadmap},
  author  = {Siyu An and Junru Lu and Junnan Dong and others},
  journal = {arXiv preprint},
  year    = {2026}
}