LLMaaD: LLM-Misdirection-as-a-Defense-Strategy

Misdirection as a Defense Against Adversarial AI Agents

Research artifact for the paper submitted to IEEE ACSAC 2026.

As generative AI is integrated into agentic applications, defenses increasingly combine model-level safeguards and external security controllers to monitor and mediate agent interactions. At the same time, attackers are adopting model-guided automation to scale multi-step probing, semantic prompt refinement, and response evaluation beyond manual effort. We analyze this setting probabilistically and derive asymptotic bounds on attacker success rate (ASR) as a function of response evaluation error rates. The analysis shows that conventional detect-and-block defense strategies can allow ASR to approach one as the attacker query budget grows. To address this limitation, we propose a detect-and-misdirect defense strategy that replaces predictable refusal responses with controlled, non-operational responses designed to corrupt the attacker's automated response evaluation process through misdirection-induced false-positives. We show that such false-positives result in a bounded asymptotic ASR. We instantiate this idea through Contextual Misdirection via Progressive Engagement (CMPE), a lightweight conversational misdirection method, and evaluate it on jailbreak benchmarks against multiple automated judge models. CMPE substantially increases attacker-side false-positives and reduces estimated ASR by up to two orders of magnitude across simulated attacker-defender judge configurations. We further evaluate CMPE against representative model-guided attack frameworks, PAIR and GPTFuzz, where it substantially reduces verified attack success rate and induces premature termination.

git clone https://github.com/Keysight/LLMaaD.git
cd LLMaaD
pip install -r requirements.txt

• vLLM (recommended for serving victim and reshaper models locally):

  pip install vllm

• Llama-Guard-3-8B for detection: served via vLLM endpoint.
• NeuralDaredevil-8B-abliterated for reshaping: served via vLLM endpoint.
• OpenAI API key for GPTFuzz and PAIR mutator/attacker model (gpt-3.5-turbo).

# Run CMPE misdirection on a single harmful prompt
python -m prompt_reshaping.cli \
  --algo algo1q \
  --victim-url http://localhost:8000/v1 \
  --victim-model vicuna-13b \
  --defender-url http://localhost:8001/v1 \
  --defender-model Llama-Guard-3-8B \
  --reshaper-url http://localhost:8002/v1 \
  --reshaper-model NeuralDaredevil-8B-abliterated \
  --mode detect-misdirect

cd llmaad_vs_adv_attacks/GPTFuzz
python gptfuzz_llmaad_parallel.py \
  --defense-mode detect-misdirect \
  --victim vicuna \
  --n-prompts 50

cd llmaad_vs_adv_attacks/JailbreakingLLMs
python run_pair_detect_misdirect_parallel.py \
  --victim vicuna \
  --n-prompts 50

After running experiments, use the post-hoc scripts to apply secondary Claude judging and export results:

export ANTHROPIC_API_KEY=<your-key>
# Optional for Azure endpoint:
# export ANTHROPIC_BASE_URL=https://<your-resource>.services.ai.azure.com/anthropic/

python llmaad_vs_adv_attacks/post_hoc/claude_judge_attack_jailbreaks.py \
  <input_results.json> <output_judged.json>

python llmaad_vs_adv_attacks/post_hoc/json_to_csv.py \
  <output_judged.json>

Use the parallel launcher from the repo root (--target selects the victim model):

# S2 — detect-and-block, vicuna target, 50 prompts
.llmaad/bin/python3 llmaad_vs_adv_attacks/autodan_results/scripts/launch_parallel.py \
    --attack turbo --mutation use_strategy --scenarios 2 --total 50 --target vicuna

# S3 — detect-and-misdirect (CMPE), vicuna target
.llmaad/bin/python3 llmaad_vs_adv_attacks/autodan_results/scripts/launch_parallel.py \
    --attack turbo --mutation use_strategy --scenarios 3 --total 50 --target vicuna

# Repeat with --target abliterated for the second victim model

Results land in llmaad_vs_adv_attacks/autodan_results/run_results/; Claude-judged finals are in llmaad_vs_adv_attacks/autodan_results/llmaad_results/turbo/.

A unified launcher covers all five PAIR script variants:

cd llmaad_vs_adv_attacks/JailbreakingLLMs

# Detect-and-block (standard judge)
python run_pair.py --mode detect-block --num-prompts 50

# Detect-and-misdirect (standard judge)
python run_pair.py --mode detect-misdirect --num-prompts 50

# Detect-and-misdirect (hardened judge)
python run_pair.py --mode detect-misdirect --judge hardened --num-prompts 50

# All modes in sequence, dry-run first
python run_pair.py --mode all --judge both --dry-run

Post-hoc validation is the same as above (Claude judge + CSV export).

Full results are in llmaad_vs_adv_attacks/final_results.md.

Positive Exits = MI FP + FP + True Pos — columns reflect the attacker's perspective; see final_results.md for full column descriptions.

Attacker: gemma1-7b-it | Scorer: openai/gpt-oss-120b | Max epochs: 20 | Break score: 8.5

All pre-computed files include Claude-judged True Positive counts that directly support the ASR = 0% claims.