Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.

Community maintained hardware plugin for vLLM on Ascend

Code for Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models

O1 Replication Journey

Ongoing research training transformer models at scale

A lightweight multilingual LLM

LangGPT: Empowering everyone to become a prompt expert! 🚀 📌 结构化提示词（Structured Prompt）提出者 📌 元提示词（Meta-Prompt）发起者 📌 最流行的提示词落地范式 | Language of GPT The pioneering framework for structured & meta-prompt…

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Inference code for Mistral and Mixtral hacked up into original Llama implementation

总结Prompt&LLM论文，开源数据&模型，AIGC应用

Benchmarking large language models' complex reasoning ability with chain-of-thought prompting

LLMs interview notes and answers:该仓库主要记录大模型（LLMs）算法工程师相关的面试题和参考答案

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Representation Engineering: A Top-Down Approach to AI Transparency

KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation

AllenAI's post-training codebase

ClickPrompt - Streamline your prompt design, with ClickPrompt, you can easily view, share, and run these prompts with just one click. ClickPrompt 用于一键轻松查看、分享和执行您的 Prompt。

Prompt 编写模式：如何将思维框架赋予机器，以设计模式的形式来思考 prompt

Best practice for training LLaMA models in Megatron-LM

Large Language Model Text Generation Inference

An Open-Source Framework for Prompt-Learning.

Repo for adapting Meta LlaMA2 in Chinese! META最新发布的LlaMA2的汉化版！ （完全开源可商用）

Chinese-LLaMA 1&2、Chinese-Falcon 基础模型；ChatFlow中文对话模型；中文OpenLLaMA模型；NLP预训练/指令微调数据集

Data and info for the paper "ParaDetox: Text Detoxification with Parallel Data"