Full Autonomy Stack for Unitree Go2

Official repository of "SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory"

VILA is a family of state-of-the-art vision language models (VLMs) for diverse multimodal AI tasks across the edge, data center, and cloud.

RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation

Vision-Language Navigation Benchmark in Isaac Lab

Low-level locomotion policy training in Isaac Lab

A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

text and image to video generation: CogVideoX (2024) and CogVideo (ICLR 2023)

Repository for Show-o, One Single Transformer to Unify Multimodal Understanding and Generation.

Video Generation, Physical Commonsense, Semantic Adherence, VideoCon-Physics

[NeurIPS 2024 Oral][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-sim…

PyTorch implementation of MAR+DiffLoss https://arxiv.org/abs/2406.11838

Open-MAGVIT2: Democratizing Autoregressive Visual Generation

Open-Sora: Democratizing Efficient Video Production for All

Exploring Efficient Fine-Grained Perception of Multimodal Large Language Models

Official code implementation of Vary: Scaling Up the Vision Vocabulary of Large Vision Language Models.

Monkey (LMM): Image Resolution and Text Label Are Important Things for Large Multi-modal Models

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

My implementation of Kosmos2.5 from the paper: "KOSMOS-2.5: A Multimodal Literate Model"

Implementation of Nougat Neural Optical Understanding for Academic Documents

Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022

A curated list of papers, datasets and resources pertaining to open vocabulary object detection.

Segment Anything in High Quality

This repo contains the code for our paper Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

Open-vocabulary Semantic Segmentation

Official implementation of the paper "Segment Everything Everywhere All at Once"

Grounded-SAM: Marrying Grounding DINO with Segment Anything & Stable Diffusion & Recognize Anything - Automatically Detect , Segment and Generate Anything