LUCID is an automated framework for auditing Large Language Model (LLM) unlearning, utilizing knowledge graph technology to generate comprehensive audit datasets for evaluating the effectiveness of LLM unlearning.
- First, clone the MUSE repository and execute the unlearning algorithm according to the official configuration:
git clone https://github.com/swj0419/muse_bench.git
cd MUSE
# Configure the environment according to MUSE official documentation
# Execute unlearning algorithm to obtain unlearned models- Install the dependency:
cd LUCID
pip install -r requirements.txtLUCID/
├── data/ # Audit data and knowledge graph data
├── metrics_utils/ # Evaluation metrics calculation tools
├── autogenkg.py # Knowledge graph generation and redundancy resolution
├── gen_question.py # Audit dataset generation
├── query_audit.py # Model querying and forgetting metrics calculation
├── run_pipe.sh # One-click script for executing the complete experimental process
├── requirements.txt # Pip dependencies.
└── README.md
This module automatically constructs knowledge graphs related to the forget set and retain set, then performs redundancy resolution to generate a clean_graph:
python autogenkg.py --ul_dataset newsGenerates comprehensive audit question sets based on the clened knowledge graph: (remember to config your own "your_api_key" for DeepSeek.)
python gen_question.py --ul_dataset newsQueries the unlearned model and calculates relevant unlearning metrics, we have provided a script in 'run_pipe.sh':
bash ./run_pipe.sh@inproceedings{jiang2026chaos,
title={From Chaos to Clarity: A Knowledge Graph-Driven Audit Dataset Generation Framework for LLM Unlearning},
author={Jiang, Weipeng and Zhai, Juan and Ma, Shiqing and Lei, Ziyan and Xue, Xiaofu and Wang, Yige and Shen, Chao},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
year={2026},
}