Public Inflection Benchmarks

Build, evaluate, understand, and fix LLM-based apps

Codebase for Context-aware Meta-learned Loss Scaling (CaMeLS). https://arxiv.org/abs/2305.15076.

[ACL'24 Outstanding] Data and code for L-Eval, a comprehensive long context language models evaluation benchmark

Repository for Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions, ACL23

The official repository for the paper "From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning".

Code and generated sounds for "Conditional Sound Generation Using Neural Discrete Time-Frequency Representation Learning", MLSP 2021

This repo hosts the code and model of "Separate What You Describe: Language-Queried Audio Source Separation", Interspeech 2022

Data and code for "DocPrompting: Generating Code by Retrieving the Docs" @ICLR 2023

An instruction-based benchmark for text improvements.

PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an implementation of kNN-LM and kNN-MT

Tevatron - Unified Document Retrieval Toolkit across Scale, Language, and Modality. Demo in SIGIR 2023, SIGIR 2025.

Data and Code Release for "On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries"

ICLR 2022 Paper, SOTA Table Pre-training Model, TAPEX: Table Pre-training via Learning a Neural SQL Executor

This repository contains source code for the TaBERT model, a pre-trained language model for learning joint representations of natural language utterances and (semi-)structured tables for semantic p…

NAACL 2019 "Structured Minimally Supervised Learning for Neural Relation Extraction"

Neural Module Network for Reasoning over Text, ICLR 2020

Graph revised convolutional network (ECML-PKDD 2020)

A Greek edition of BERT pre-trained language model

GraphParser is a semantic parser which can convert natural language sentences to logical forms and graphs.

KnowBert -- Knowledge Enhanced Contextual Word Representations

Google Drive direct download of big files

Code for using and evaluating SpanBERT.

Universal Adversarial Triggers for Attacking and Analyzing NLP (EMNLP 2019)

PyTorch implementation of A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text (EMNLP 2019)

Python scripts preprocessing Penn Treebank and Chinese Treebank

Neural network toolkit for sentence pair modeling.

A tool for holistic analysis of language generations systems

Code for CVPR'19 "Recursive Visual Attention in Visual Dialog"