A framework for identifying vulnerabilities in and evaluating the security of AI systems.
Full Documentations: https://avise.readthedocs.io
- Python 3.10+
- Docker (for running models backend)
- pip
git clone https://github.com/ouspg/AVISE.git
cd AVISE# Create virtual environment
python -m venv venv
source venv/bin/activate # Or venv/Scripts/Activate on Windows
# Install dependencies
pip install -r requirements.txtGPU Version:
docker-compose -f docker/ollama/docker-compose.yml up -dCPU-only Version:
docker-compose -f docker/ollama/docker-compose-cpu.yml up -dAfter Ollama is running, pull the models you want to test:
# Pull models for testing and for evaluation
docker exec -it avise-ollama ollama pull <model_name>Edit avise/configs/connector/ollama.json:
{
"target_model": {
"connector": "ollama-lm",
"type": "language_model",
"name": "<NAME_OF_TARGET_MODEL>",
"api_url": "http://localhost:11434", #Ollama default
"api_key": null
},
"eval_model": {
"connector": "ollama-lm",
"type": "language_model",
"name": "<NAME_OF_EVALUATION_MODEL>", #Optional
"api_url": "http://localhost:11434", #Ollama default
"api_key": null
}
}python -m avise --SET <SET_name> --connectorconf <path> --SETconf <path> [options]| Argument | Description |
|---|---|
--SET |
Security Evaluation Test to run (e.g., prompt_injection, context_test) |
--connectorconf |
Path to Connector configuration JSON |
--SETconf |
Path to test configuration JSON |
| Argument | Description |
|---|---|
--format, -f |
Report format: json, html, md |
--output |
Custom output file path |
--reports-dir |
Base directory for reports (default: reports/) |
--list |
List available tests and formats |
-verbose |
Enable verbose logging |
-version |
Print version |