Best practices for training DeepSeek, Mixtral, Qwen and other MoE models using Megatron Core.

Bridge Megatron-Core to Hugging Face/Reinforcement Learning

Build 3D Gaussian Splatting from scratch with NVIDIA Warp in Python — CPU/GPU compatible, with a clean and minimalist design focused on learning modern graphics.

Must-read papers and blogs about parametric knowledge mechanism in LLMs.

A Comprehensive Survey on Long Context Language Modeling

MPO: Boosting LLM Agents with Meta Plan Optimization (EMNLP 2025 Findings)

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

An awesome repository & A comprehensive survey on interpretability of LLM attention heads.

A collection of LLM papers, blogs, and projects, with a focus on OpenAI o1 🍓 and reasoning techniques.

A repo for open resources & information for people to succeed in PhD in CS & career in AI / NLP

Code and data used in the paper: "Training on Incorrect Synthetic Data via RL Scales LLM Math Reasoning Eight-Fold"

Sparsify transformers with SAEs and transcoders

A curated list of Large Language Model (LLM) Interpretability resources.

🙌 OpenHands: Code Less, Make More

GitHub最新hosts。解决GitHub图片无法显示，加速GitHub网页浏览。

[ACL 2024]Official GitHub repo for OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems.

A minimal GPU design in Verilog to learn how GPUs work from the ground up

LongEmbed: Extending Embedding Models for Long Context Retrieval (EMNLP 2024)

The official Meta Llama 3 GitHub site

Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024

An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray (PPO & GRPO & REINFORCE++ & vLLM & Ray & Dynamic Sampling & Async Agentic RL)

WebAssembly (Wasm) Build and Bindings for llama.cpp

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

State-of-the-art bilingual open-sourced Math reasoning LLMs.

Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.

LLMs as Copilots for Theorem Proving in Lean

A Production-ready Reinforcement Learning AI Agent Library brought by the Applied Reinforcement Learning team at Meta.