Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image …

collection of sample scrips for HPC Mahameru

Data Valuation on In-Context Examples

ACL Findings '23 - ICE: Multi-dimensional Evaluation of Text Summarization with In-Context Learning

An Open-Source Framework for Prompt-Learning.

A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers)

Implementation of ChatGPT RLHF (Reinforcement Learning with Human Feedback) on any generation model in huggingface's transformer (blommz-176B/bloom/gpt/bart/T5/MetaICL)

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

Train transformer language models with reinforcement learning.

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

A Great Collection of Deep Learning Tutorials and Repositories

⚡ Building applications with LLMs through composability ⚡

A collection of open-source dataset to train instruction-following LLMs (ChatGPT,LLaMA,Alpaca)

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Code for "Learning to summarize from human feedback"

Course for Interpreting ML Models

EMNLP 2022: Analyzing and Evaluating Faithfulness in Dialogue Summarization

Code for ACL 2020 paper: USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation (https://arxiv.org/pdf/2005.00456)

Repository for EMNLP 2022 Paper: Towards a Unified Multi-Dimensional Evaluator for Text Generation

Code for the EMNLP 2022 paper: Near-Negative Distinction

EMNLP 2021 - CTC: A Unified Framework for Evaluating Natural Language Generation

Diffusion-LM

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.