Skip to content

Maxamed/llm-contract

Repository files navigation

llm-contracts

PyPI version Python License Tests Build Status

🛡️ "ESLint + Pytest" for AI responses — Catch LLM mistakes before they reach production.
Schema validation, content linting, and professional reports for any LLM.


📦 Install

pip install llm-contracts

⚡ Why llm-contracts?

LLMs are fluent, confident, and totally wrong just often enough to break your app.

  • Air Canada's chatbot promised non-existent bereavement fares → Legal action
  • CNET's AI published financial advice with wrong interest rates → Public corrections
  • ChatGPT lawyer submitted fake legal citations in court → Professional sanctions

llm-contracts validates every AI response before it causes problems.


🚀 Quick Start

CLI

# Validate AI output against schema
llm-validate output.json --schema schema.yaml

# Generate professional reports
llm-validate output.json --schema schema.yaml --html-report report.html

Python API

from llm_contracts import contracts

# Validate output against schema
result = contracts.validate(data, "schema.yaml")

if not result.is_valid:
    print("AI failed validation:")
    for error in result.errors:
        print(f"  - {error}")

# Generate reports
contracts.generate_report(result, "report.html", "schema.yaml", format="html")

✅ Key Features

  • Schema Validation — Ensure proper JSON/YAML structure and data types
  • Content Linting — Check keywords, tone, word count, patterns, quality rules
  • Professional Reports — Beautiful HTML and Markdown validation reports
  • Framework Agnostic — Works with OpenAI, Anthropic, local models, any LLM
  • CLI + Python API — Use in scripts or integrate into applications
  • Zero Vendor Lock-in — Pure validation logic, no external API calls required

📋 Example Schema

schema:
  title:
    type: str
    min_length: 10
  description:
    type: str
    min_length: 100

rules:
  - keyword_must_include: ["quality", "premium"]
  - keyword_must_not_include: ["cheap", "defective"]
  - no_placeholder_text: "\\[YOUR_TEXT_HERE\\]"
  - word_count_min: 100
  - word_count_max: 500

Result: Every AI response gets validated before reaching your users. No more silent failures.


📚 Documentation & Links


🤝 Contributors

Created by Mohamed Jama - Built for developers shipping AI features in production.

Major Contributors:

  • Abdirahman Attila - Frontend web interface, comprehensive documentation website, enhanced testing suite, and Jekyll/GitHub Pages setup

We welcome contributions! See CONTRIBUTING.md for guidelines.


🏷 License

MIT © Mohamed Jama - see LICENSE file for details.


Solidarity

I believe developers don't live outside the world — we help shape it.

I stand in full solidarity with the people of Palestine, especially in Gaza, who continue to endure conditions no human should face: siege, displacement, and the systematic erasure of life and dignity.

This is not about politics. It's about humanity.

Silence is not neutrality.

Learn more and support:


Stop hoping your AI gets it right. Start knowing it does.

About

Validate, lint, and assert correctness of LLM-generated outputs. Like ESLint for AI.

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages