Training Large Language Model to Reason in a Continuous Latent Space

Open Source Deep Research Alternative to Reason and Search on Private Data. Written in Python.

Generative Representational Instruction Tuning

SGPT: GPT Sentence Embeddings for Semantic Search

AllenAI's post-training codebase

Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipy

LOFT: A 1 Million+ Token Long-Context Benchmark

[Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers

OpenResearcher, an advanced Scientific Research Assistant

One-stop shop for running and fine-tuning transformer-based language models for retrieval

State-of-the-Art Text Embeddings

Source code of our paper "PairDistill: Pairwise Relevance Distillation for Dense Retrieval", EMNLP 2024 Main.

TrustRAG：The RAG Framework within Reliable input,Trusted output

Unified Learned Sparse Retrieval Framework

RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking.

[EMNLP 2024] A Retrieval Benchmark for Scientific Literature Search

A Workbench for Autograding Retrieve/Generate Systems

Repository for "MultiHop-RAG: A Dataset for Evaluating Retrieval-Augmented Generation Across Documents" (COLM 2024)

final code for fultr

A modular RL library to fine-tune language models to human preferences

[ACL 2024] Long-Context Language Modeling with Parallel Encodings

Ollama Python library

Stanford NLP Python library for Representation Finetuning (ReFT)

Source code of the paper: RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering [Findings of ACL 2024]

RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation.

This includes the original implementation of SELF-RAG: Learning to Retrieve, Generate and Critique through self-reflection by Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi.