SynVAR: Synergizing Spatial and Semantic Alignment in Visual Autoregressive Model

Visual autoregressive (VAR) models have gained widespread popularity through next-scale prediction, yet they still struggle with complex scenes containing multiple objects and attributes. Existing diffusion-oriented enhancement methods do not adequately address the cross-scale propagation and accumulation of errors in VAR. We propose SynVAR, the first training-free enhancement framework tailored to the VAR paradigm. Its spatial-semantic collaborative control suppresses propagation errors and improves complex-scene generation. Extensive quantitative and qualitative experiments demonstrate consistent improvements in spatial composition, semantic fidelity, and visual quality.

• Global Guidance (GG): establishes a coherent spatial structure at early scales.
• Receptive Field Constraint (RFC): prevents cross-region semantic confusion.
• High-Frequency Compensation (HFC): restores fine-grained details lost across scales.

python examples/demo.py --backend infinity --checkpoint-dir /path/to/infinity-checkpoints --case all

python examples/demo.py --backend switti --checkpoint-dir /path/to/switti-checkpoints --case all

Tested with Python 3.12, PyTorch 2.6.0, torchvision 0.21.0, CUDA 12.4, and FlashAttention 2.7.4.post1.

git clone https://github.com/NJU-PCALab/SynVAR.git
cd SynVAR
python -m pip install torch==2.6.0 torchvision==0.21.0 \
  --index-url https://download.pytorch.org/whl/cu124
python -m pip install flash-attn==2.7.4.post1 --no-build-isolation
python -m pip install -e .

The modified Infinity and Switti inference sources are included in this repository; no patching or second source checkout is required. Model checkpoints remain external.

Point --checkpoint-dir at a directory with the corresponding layout:

infinity-checkpoints/
├── infinity_2b_reg.pth
├── infinity_vae_d32reg.pth
└── flan-t5-xl/

switti-checkpoints/
├── switti-1024/
├── vqvae-switti/
├── clip-vit-large-patch14/
└── clip-vit-bigG/

List the bundled examples without loading a model:

python examples/demo.py --backend infinity --list
python examples/demo.py --backend switti --list

Run the teaser dog/balloon example:

python examples/demo.py --backend infinity \
  --checkpoint-dir /path/to/infinity-checkpoints \
  --case 0 --compare

Run a Switti example:

python examples/demo.py --backend switti \
  --checkpoint-dir /path/to/switti-checkpoints \
  --case 0 --compare

--case accepts the numeric index shown by --list or all. The scene plans and runtime values are defined in examples/demo_cases.json. --case all renders every example for one backend with one model load.

Each case directory contains synvar.png and run.json. With --compare, it also contains baseline.png and a side-by-side comparison.png. The output root contains summary.json with the backend, native revision, and sampling configuration.

The demos use CFG 4 and temperature 0.5 for Infinity. Switti uses CFG 4, top-k 900, top-p 0.97, CFG at stages 0..10, temperature 0.1 from stage 11 onward, and float32. These backend-specific sampling settings do not change the shared GG/RFC/HFC configuration.

python examples/generate.py \
  --model-path /path/to/infinity_2b_reg.pth \
  --vae-path /path/to/infinity_vae_d32reg.pth \
  --text-encoder /path/to/flan-t5-xl \
  --plan examples/two_objects.json \
  --output output.png

A plan contains a global prompt and normalized regions:

{
  "prompt": "a red apple on the left and a blue bowl on the right",
  "regions": [
    {"concept": "red apple", "xyxy": [0.0, 0.0, 0.5, 1.0]},
    {"concept": "blue bowl", "xyxy": [0.5, 0.0, 1.0, 1.0]}
  ]
}

from pathlib import Path

from synvar import InfinityBackend, InfinitySamplingConfig, Region, ScenePlan, SynVARConfig

backend = InfinityBackend.from_pretrained(
    model_path=Path("/path/to/infinity_2b_reg.pth"),
    vae_path=Path("/path/to/infinity_vae_d32reg.pth"),
    text_encoder=Path("/path/to/flan-t5-xl"),
)
plan = ScenePlan(
    prompt="a dog above a balloon",
    regions=(
        Region("a dog", 0.0, 0.0, 1.0, 0.5),
        Region("a balloon", 0.0, 0.5, 1.0, 1.0),
    ),
)
image = backend.generate(
    plan,
    config=SynVARConfig(),
    sampling=InfinitySamplingConfig(cfg=4.0, tau=0.5),
)

The same ScenePlan works with the Switti backend:

from synvar import SwittiBackend, SwittiSamplingConfig, switti_default_config

backend = SwittiBackend.from_pretrained(
    model_path=Path("/path/to/switti-1024"),
    vae_path=Path("/path/to/vqvae-switti"),
    text_encoder=Path("/path/to/clip-vit-large-patch14"),
    text_encoder_2=Path("/path/to/clip-vit-bigG"),
)
image = backend.generate(
    plan,
    config=switti_default_config(),
    sampling=SwittiSamplingConfig(
        cfg=4.0,
        top_k=900,
        top_p=0.97,
        turn_on_cfg_start_si=0,
        turn_off_cfg_start_si=11,
        last_scale_temp=0.1,
    ),
)

@article{chen2026synvar,
  title={SynVAR: Synergizing Spatial and Semantic Alignment in Visual Autoregressive Model},
  author={Chen, Zhennan and Shi, Tianxing and Xu, Pengcheng and Nan, Kepan and Wang, Qian and Yi, Zili and Yang, Jian and Tai, Ying},
  journal={arXiv preprint arXiv:2608.07948},
  year={2026}
}