#3636th of 41 in Gateways and observability
ragas
Python library for scoring LLM and RAG application outputs with metrics
- Stars
- 16.1k
- License
- Apache-2.0
- Last commit
- Feb 2026
- Last release
- Jan 2026
- Language
- Python
12,998 on 26 Mar 202616,136 on 22 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Ragas is a Python package for evaluating LLM applications. It ships pre-built and custom metrics, including LLM-judged ones such as DiscreteMetric, and can generate test datasets when none exist. It integrates with LangChain and observability tools, and a `ragas quickstart rag_eval` command scaffolds an example RAG evaluation project.
Who it is for: Developers measuring the quality of LLM and RAG apps in Python
Strengths
- Installs with pip; quickstart command scaffolds a RAG evaluation project
- Custom metrics defined by a prompt and allowed values, scored asynchronously
- Generates test datasets when none are available
- Apache-2.0 licensed; telemetry can be disabled with RAGAS_DO_NOT_TRACK
Weaknesses
- LLM-based metrics need an external model, such as an OpenAI key in the example
- Only the rag_eval quickstart template exists; other templates are listed as coming soon
- Collects anonymized usage analytics by default; opt-out required
- No UI or hosted dashboard described in the README
What it needs
- no GPU
- Needs LLM provider API (OpenAI in example)
- Models: OpenAI, LangChain
Quick answers
Is ragas open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Does ragas need a GPU?
No, it runs without one.
Also in Gateways and observability
See all 41| Rank | Project | Score |
|---|---|---|
| 1 | OmniRouteLLM gateway routing one OpenAI-compatible endpoint across 372 providers | 96 out of 100 |
| 2 | new-apiGateway that unifies LLM providers behind OpenAI, Anthropic and Gemini APIs | 90 out of 100 |
| 3 | MLflowTracing, evals, prompt registry and AI gateway plus classic ML tracking | 89 out of 100 |
| #4 | LiteLLMOpenAI-format gateway and Python SDK for calling 100+ LLM providers | 86 out of 100 |
| #5 | freellmapiOpenAI-compatible gateway pooling free-tier LLM providers with automatic failover | 86 out of 100 |