A comprehensive, self-paced training curriculum for security researchers focused on red teaming GenAI and AI/ML systems. This repository contains hands-on labs, theoretical content, and practical demonstrations of adversarial techniques.

Red teaming for AI/ML goes far beyond traditional RAI (Responsible AI) testing. While RAI focuses on fairness, bias, and ethical considerations, red teaming encompasses:

• Adversarial Attacks: Crafting inputs to fool models or extract sensitive information
• Security Vulnerabilities: Identifying exploitable weaknesses in AI systems
• Privacy Breaches: Testing for data leakage and membership inference
• Model Manipulation: Poisoning, backdoors, and supply chain attacks
• Evasion Techniques: Bypassing safety guardrails and content filters
• System-Level Exploits: Prompt injection, jailbreaking, and serialization attacks

This training teaches offensive security techniques for AI systems, the same methods attackers use, so you can better defend against them.

New to AI Security? Start with our companion resource: GenAI Essentials - LLM Security Notebook for foundational concepts before diving into this advanced curriculum.

Security researchers with:

• Intermediate to advanced technical background
• Understanding of machine learning fundamentals
• Interest in AI/ML security and adversarial techniques

The training is organized into sequential modules, each building upon previous concepts:

• Overview of AI/ML security landscape
• Understanding LLM architecture and vulnerabilities
• Setting up your red teaming environment
• Introduction to adversarial thinking

• Understanding prompt injection attacks
• Jailbreaking techniques and methodologies
• Defense mechanisms and their limitations
• Hands-on labs with real-world scenarios

• White-box evasion techniques
• Black-box evasion strategies
• Adversarial example generation
• Practical exercises with various models

• Training data inference attacks
• Model inversion techniques
• Membership inference attacks
• Privacy-preserving defenses

• Training data poisoning
• Backdoor attacks
• Supply chain vulnerabilities
• Detection and mitigation strategies

• Model weight extraction
• Model distillation attacks
• Self-instruct data generation
• Serialization vulnerabilities

• Comprehensive security assessment methodologies
• Automated testing frameworks
• Red team exercise scenarios
• Reporting and remediation

• Capstone project
• Real-world scenario testing
• Comprehensive evaluation

• Python 3.12+
• Basic understanding of machine learning
• Familiarity with Jupyter notebooks
• Access to GPU (recommended for some exercises)

# Clone the repository
git clone <repository-url>
cd genai-security-training

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install --upgrade pip
pip install -r requirements.txt

# Verify installation (Optional and note all packages)
python -c "import torch, transformers, numpy, pandas; print('✅ Ready!')"

Note: Installation takes 10-15 minutes. Labs that need additional packages (like textattack, adversarial-robustness-toolbox, etc.) will automatically install them when you run the notebook.

1. Start with Module 1 to understand the foundations
2. Progress sequentially through each module
3. Complete hands-on labs before moving to the next module
4. Reference materials are provided throughout
5. Final assessment tests comprehensive understanding

Each module contains:

• Theory: Markdown documents explaining concepts
• Labs: Jupyter notebooks with hands-on exercises
• References: Links to papers, tools, and additional resources
• Assessments: Knowledge checks and practical exercises

This course uses industry-standard security testing tools:

• Adversarial Robustness Toolbox (ART) - IBM's comprehensive library for adversarial ML (Module 7)
• TextAttack - NLP-focused adversarial attack framework (Modules 3, 7)
• SHAP - Model explainability and robustness testing (Module 7)

• OWASP LLM Top 10 - Top vulnerabilities in LLM applications
• NIST AI Risk Management Framework - AI risk assessment guidelines
• MITRE ATLAS - Adversarial threat landscape for AI systems

• GenAI Essentials - Foundational GenAI concepts and security basics
• LLM Security Notebook - Essential primer on LLM security concepts (recommended prerequisite)
• Artificial Diaries - Research publications and case studies
• GitHub: @schwartz1375

This training is designed for:

• Security research and testing
• Defensive security improvements
• Educational purposes

NOT for:

• Malicious attacks on production systems
• Unauthorized testing
• Illegal activities

Always obtain proper authorization before testing any system.

• 8 Complete Modules - Introduction through Capstone
• 40 Jupyter Notebooks - Hands-on labs, interactive theory, and solutions
• 29 Theory & Documentation Files - Comprehensive markdown guides and module overviews
• Industry-Standard Tools - Integrated ART, TextAttack, and SHAP frameworks in notebooks
• Device Detection Code - Built-in CUDA/Apple Silicon MPS/CPU detection in labs

Note: Labs include device detection code to automatically use your GPU (NVIDIA CUDA or Apple Silicon MPS) when available.

• QUICK_START.md - Get started in 15 minutes
• INSTRUCTOR_GUIDE.md - Complete teaching guide with answer key locations and grading rubrics
• Module ANSWERS.ipynb - Runnable Jupyter notebooks with complete solutions (20+ exercises) in modules 1-5 and 7 labs folders