This project uses pytest for unit testing. All test cases are under the test/ directory.

• Recommended Python 3.10 or above.
• Install dependencies:

  pip install -r ../requirements.txt
  pip install pytest

Tests are split into three tiers, then by hardware backend (test_cpu/, test_cuda/, test_hpu/, test_xpu/, test_ark/, test_mlx/):

• unit/ — fast, self-contained tests. Run on every PR (unit-test*.yml).
• integration/ — tests against third-party frameworks (vLLM, SGLang, LLMCompressor, INC, HuggingFace). Run nightly — CPU via nightly-test-cpu.yml, XPU via nightly-test-xpu.yml, CUDA via nightly-test-cuda.yml.
• e2e/ — full-model / real inference-engine tests. Run manually

Within unit/test_cpu/ and unit/test_cuda/, tests are grouped by functionality:

• core/ - Core AutoRound API and quantization workflows
• quantization/ - Quantization techniques (mixed-bit, MXFP, NVFP4, activation quant)
• export/ - Model serialization (GGUF, AutoGPTQ, AutoRound format)
• backends/ - Inference backends (Torch, Marlin, Triton, ExLlamaV2)
• models/ - Architecture-specific tests (MLLMs, VLMs, MoE, Diffusion, Omni)
• schemes/ - Quantization scheme selection and configuration
• utils/ - Calibration datasets, logging, CLI, model loading
• advanced/ - Multi-GPU, FP8 input, custom pipelines

Pytest configuration file that:

• Adds parent directory to sys.path for easy debugging without installation
• Defines HPU-specific test options (--mode=compile/lazy)
• Imports all fixtures from fixtures.py

Provides reusable pytest fixtures for testing:

Model Fixtures:

• tiny_opt_model_path - OPT-125M model with 2 layers (session scope)
• tiny_qwen_model_path - Qwen-0.6B model with 2 layers
• tiny_lamini_model_path - LaMini-GPT-124M with 2 layers
• tiny_gptj_model_path - Tiny GPT-J model
• tiny_phi2_model_path - Phi-2 model with 2 layers
• tiny_deepseek_v2_model_path - DeepSeek-V2-Lite with 2 layers
• tiny_qwen_moe_model_path - Qwen-1.5-MoE with 2 layers
• tiny_qwen_vl_model_path - Qwen2-VL-2B with 2 layers (vision model)
• tiny_qwen_2_5_vl_model_path - Qwen2.5-VL-3B with 2 layers

Data Fixtures:

• dataloader - Simple calibration dataloader with 4 text samples

All model fixtures:

• Use session scope to avoid reloading models for each test
• Automatically save tiny models to ./tmp/ directory
• Clean up temporary files after test session ends

Utility functions for testing:

Model Path Resolution:

get_model_path(model_name)  # Automatically finds local or remote model path

Predefined Model Paths:

opt_name_or_path  # facebook/opt-125m
qwen_name_or_path  # Qwen/Qwen3-0.6B
lamini_name_or_path  # MBZUAI/LaMini-GPT-124M
qwen_vl_name_or_path  # Qwen/Qwen2-VL-2B-Instruct
# ... and more

Model Manipulation:

get_tiny_model(model_path, num_layers=2)  # Create tiny model by slicing layers
save_tiny_model(model_path, save_path)  # Save tiny model to disk

Model Inference:

model_infer(model, tokenizer, input_text)  # Run inference and return output

Data Utilities:

DataLoader()  # Simple dataloader for calibration datasets

# unit/test_cpu/quantization/test_new_method.py
import pytest
from auto_round import AutoRound
from test.helpers import opt_name_or_path


class TestNewQuantMethod:
    def test_quantization(self, tiny_opt_model_path, dataloader):
        """Test new quantization method."""
        autoround = AutoRound(model=tiny_opt_model_path, bits=4, group_size=128, iters=2, dataset=dataloader)
        autoround.quantize()
        assert autoround is not None

from test.helpers import model_infer, opt_name_or_path, get_model_path


def test_model_inference(tiny_opt_model_path):
    # Use predefined model path
    model_name = opt_name_or_path

    # Or resolve custom model path
    custom_model = get_model_path("custom/model-name")

    # Run inference using helper
    from transformers import AutoModelForCausalLM, AutoTokenizer

    model = AutoModelForCausalLM.from_pretrained(tiny_opt_model_path)
    tokenizer = AutoTokenizer.from_pretrained(tiny_opt_model_path)
    output = model_infer(model, tokenizer, "Hello world")

• Fast & self-contained → unit/test_<hw>/<category>/
• Needs a third-party framework (vLLM, SGLang, LLMC, INC) → integration/test_<hw>/
• Full model / real inference engine → e2e/test_<hw>/
• CPU-specific → */test_cpu/, CUDA-specific → */test_cuda/
• Import from parent: from test.helpers import ...

• For long-running or non-critical tests under test_cuda/, use @pytest.mark.skip_ci(reason="...") and provide a clear reason why the test should not run in CI.
• In CI, each test function has a default timeout of 30 seconds, while each test file has a timeout of 10 minutes.
• Prefer simplifying a test so that it completes within the default timeout. If the test cannot be reduced further, extend its timeout with @pytest.mark.timeout(seconds). If a test file exceeds the 10-minute limit, split its tests into smaller files whenever possible.

@pytest.mark.timeout(120)
def test_long_running_case(): ...


@pytest.mark.skip_ci(reason="Time-consuming accuracy evaluation; covered by nightly tests")
def test_optional_accuracy_evaluation(): ...

# Run all fast unit tests (the default `testpaths` in pytest.ini)
pytest

# Run a specific tier / hardware
pytest unit/test_cpu/
pytest integration/test_cpu/
pytest e2e/test_cpu/

# Run specific category
pytest unit/test_cpu/quantization/

# Run specific file
pytest unit/test_cpu/core/test_autoround.py

# Run specific test
pytest -k "test_layer_config"

# Run with verbose output
pytest -v -s

• unit/test_cpu/: Install pip install -r unit/test_cpu/requirements.txt
• unit/test_cuda/: Install pip install -r unit/test_cuda/requirements.txt
  • VLM: pip install -r unit/test_cuda/requirements_vlm.txt
  • Diffusion: pip install -r unit/test_cuda/requirements_diffusion.txt
  • LLMC: pip install -r unit/test_cuda/requirements_llmc.txt
  • SGLang: pip install -r unit/test_cuda/requirements_sglang.txt

When adding new tests:

1. Place in appropriate category subdirectory
2. Use existing fixtures and helpers
3. Clean up resources in teardown methods
4. Use descriptive names and docstrings

For questions, open an issue.