A single CLAUDE.md file to improve Claude Code behavior, derived from Andrej Karpathy's observations on LLM coding pitfalls.

inference-hive is a toolkit to run distributed LLM inference on SLURM clusters. Configure a few cluster, inference server and data settings, and scale your inference workload across thousands of GPUs.

Crowd-sourced lists of urls to help Common Crawl crawl under-resourced languages. See https://github.com/commoncrawl/web-languages-code/ for the code

Web Content Extraction Benchmark

Utility to fetch Open PageRank scores for many domains at once, with caching, batching, and API quota handling.

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

The project introduces a machine learning framework for automatically evaluating the trustworthiness of online domains using link-based and domain-level features. It predicts credibility scores for…

global news data collection

DataComp for Language Models

These notebooks analyze daily trends in online news coverage, examining news volume, topic distribution, source reliability, disinformation tactics, check-worthy claims, and visual-text alignment.

Persuasion-Augmented Chain-of-Thought for Disinformation Detection

In-depth study of the graphrag

Repository for the companion Colab notebook of the Domain-Specific Small Language Models book.

Design and evaluate search engine interventions for safe and fair search rankings.

Process Common Crawl data with Python and Spark

Granite Snack Cookbook -- easily consumable recipes (python notebooks) that showcase the capabilities of the Granite models

news-please - an integrated web crawler and information extractor for news that just works

Codes and datasets for the paper Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse

dblp Knowledge Graph

Cache-Augmented Generation

A graph analysis of the Dell Research Newswire Dataset, specifically focused on characterizing newspaper similarities and behaviors

Code and models for NExT-GPT: Any-to-Any Multimodal Large Language Model

"LightRAG: Simple and Fast Retrieval-Augmented Generation"