A curated list of awesome responsible machine learning resources.

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

AI Agent Governance Toolkit — Policy enforcement, zero-trust identity, execution sandboxing, and reliability engineering for autonomous AI agents. Covers 10/10 OWASP Agentic Top 10.

Secrets of RLHF in Large Language Models Part I: PPO

UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection

Internal Safety Collapse: Turning the LLM or an AI Agent into a sensitive data generator.

A resource repository for machine unlearning in large language models

Deliver safe & effective language models

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022

The open agent control plane. Govern autonomous AI agents with pre-execution policy enforcement, approval gates, and audit trails. Works with LangChain, CrewAI, MCP, and any framework.

Stop AI agents from doing things you didn't ask for.

The open-source diagnostic for AI misalignment. 32 tests across fabrication, manipulation, deception, unpredictability, and opacity. Provider-agnostic. Runs against OpenAI, Anthropic, Bedrock, Azure, Gemini, and more. Letter grade in under 5 minutes, content-addressed manifest for bit-identical replay. Built by iMe.

Open Source LLM toolkit to build trustworthy LLM applications. TigerArmor (AI safety), TigerRAG (embedding, RAG), TigerTune (fine-tuning)

Runtime policy enforcement for AI agents. Cryptographic audit trail, human-in-the-loop approvals, kill switch. Zero code changes.

Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours

Aligning AI With Shared Human Values (ICLR 2021)

🦀 Prevents outdated Rust code suggestions from AI assistants. This MCP server fetches current crate docs, uses embeddings/LLMs, and provides accurate context via a tool call.

A self-hosted AI infrastructure for private RAG and multi-model applications.

[NeurIPS '23 Spotlight] Thought Cloning: Learning to Think while Acting by Imitating Human Thinking

An unrestricted attack based on diffusion models that can achieve both good transferability and imperceptibility.