🧪⚡ Lightweight, hackable multi-agent orchestration lab. YAML configs, CLI + Python API, 10 presets, transcripts, checkpoints. Perfect for LLM experiments!

👹 YETi: The abominable snowman of YouTube A/B testing - statistically significant, AI-powered, and surprisingly friendly.

ReSimHub is a scalable research-grade backend framework providing REST + async APIs for reinforcement learning (RL) experimentation, simulation environments, training orchestration, and benchmarking.

A/B test lift simulation & analysis in Python: CUPED variance reduction, power analysis, and guardrail metrics with reproducible pytest CI.

This app shows the use of bayesian statistics for AB Testing using the beta distribution.

GraphRAG-powered Small Language Model for experimentation and statistical analysis Q&A

Modern A/B experimentation utilities: CUPED/CUPAC hooks, triggered analysis, SRM, switchback helpers, and power sims.

Implementation of metaheuristic optimization methods in Python for scientific, industrial, and educational scenarios. Experiments can be executed in parallel or in a distributed fashion. Experimental results can be evaluated in various ways, including diagrams, tables, and export to Excel.

Generic functions for experiment analysis and design

Experimenter Analytics Tools — A/B testing platform for analysts and data scientists who need full control over their experimentation workflow.

End-to-end A/B testing pipeline for e-commerce checkout optimization. Simulates realistic user journeys, runs statistical tests, and provides interactive analytics dashboards.

Simple development environment to quickly explore recent Deep Learning algorithms, plus some experiments and results. | Explored: Foundational Models for Autolabelling; Chat LLMs+RAG; Diffusion for 2D 3D data generation; Docker DevEnv + CI/CD

Geographic Experiment design and Evaluation Tool designed to help you determine the true lift of marketing efforts.

Tool for the automatic orchestration of experiments targeting software systems

Tools to create experiment designs

Auditory experiment framework with a focus on rapid experiment design

RLHF-Blender: A Configurable Interactive Interface for Learning from Diverse Human Feedback

🧪 AB Lab is an end-to-end A/B testing toolkit built in Python with an interactive Streamlit interface. It enables researchers, data scientists, and product teams to design, simulate, and analyze experiments—from power calculations and bias reduction to frequentist and Bayesian inference.

[QDB'25] Out in the wild of modern data science, good tabular AI meets bad data. Who wins?

A Django app library for experimentation.