A multimodal decision model inspired by Jev — text, images and video in; decision probabilities out.

English · 简体中文
Intro · News · Demos · Models & datasets · Eval results · Quick start

Valen (万澜) brings visual perception to System One decision-making. Inspired by Jev, it evaluates text, images and video against task instructions and returns probabilities over supplied candidates, giving software a structured decision interface. The repository includes the model implementation, data processing, SFT and experimental RLCD training, and inference and evaluation commands, with support for training on your own data.

• 2026-10-07: Faster, more accurate, more versatile. Released the latest multimodal System One model Valen (0.8B, 2B and 4B), supporting text, images and video, and the VisualDecisionBench benchmark.
• 2026-09-30: Training&Inference Support: Added Flash Attention 2 support, enabled training across multiple nodes and GPUs, and improved GPU memory utilization during training.
• 2026-09-23: Open-sourced the Valen repository and released Valen-preview-0923 with its corresponding training data.

Valen image and video decision-making — click the preview to watch the full demo.

Game decision-making — five successful Valen rollouts. Click any preview to watch the full video.

Sokoban Plans a 13-step optimal solution.		Frozen Lake 11-step optimal safe route.		Maze Completes a 22-step long-horizon route.
Lane Racer Changes lanes and clears the course in 10 steps.			Parkour Runner Chains jumps, dashes, ducks and runs over 14 steps.

VisualDecisionBench contains Image and Video subsets for visual decision evaluation. Earlier releases include General 100k for training, General 5k for evaluation, and Sokoban for game training and evaluation.

Visual decisions at 0.8B, 2B and 4B. Valen uses Qwen3.5 + Mixer with shared-state inference, trained with SFT on millions of samples. The evaluation compares these checkpoints with published decision models on VisualDecisionBench-Image, VisualDecisionBench-Video and public JevBench.

Earlier results remain in the archived README.

Install the dependencies. Run hf auth login if the Hugging Face repositories require access.

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "Valen-Team/Valen-2B", trust_remote_code=True, dtype="auto",
    attn_implementation="sdpa",
).to("cuda").eval()
torch.set_float32_matmul_precision("highest")
torch.backends.cudnn.allow_tf32 = False

print(model.predict({
    "state": "A cat is on the sofa.",
    "questions": {
        "animal": {"type": "choice", "instructions": "Which animal is present?",
                   "criteria": {"cat": "A cat", "dog": "A dog"}},
        "on_sofa": {"type": "noul", "instructions": "The cat is on the sofa."},
    },
}, execution="shared_state"))

Use execution="question" for separate questions or attn_implementation="flash_attention_2" with Flash Attention installed. Image/video inputs follow the data format.

Download and unpack the media, then run the evaluation script:

hf download Valen-Team/VisualDecisionBench --repo-type dataset --local-dir data/VisualDecisionBench
python data/VisualDecisionBench/unpack_assets.py
python -m evaluation.visualdecisionbench.evaluate \
  --model Valen-Team/Valen-2B --data data/VisualDecisionBench \
  --output output/visualdecisionbench

Add --lora --revision lora-ckpt to evaluate the earlier LoRA model. Outputs: predictions.jsonl and metrics.json, including image/video and Choice/Noul/Score breakdowns and elapsed time. Videos use 16 frames; accuracy uses hard labels, while soft-label Score questions contribute probability metrics.

The shared-state recipes use Mixer. Install Flash Attention 2 or set model.attn_implementation to sdpa. Set your data paths, output directories and training budget in the configs; the defaults use smoke data and a 100-step limit.

python scripts/setup/prepare_model.py

# Two-stage SFT: head warmup, then joint LoRA + Mixer + visual merger.
python -m valen.train --config configs/train/qwen/sft_shared_state_warmup.json
python -m valen.train --config configs/train/qwen/sft_shared_state_joint.json \
  --initialize output/qwen-shared-state/warmup/latest

# RLCD: candidate-action RL initialized from the SFT checkpoint.
python -m valen.train --config configs/train/qwen/rlcd_shared_state_joint.json \
  --initialize output/qwen-shared-state/joint/latest

For full training, keep the warmup stage and replace the joint SFT recipe with sft_shared_state_full_joint.json, using the same --initialize checkpoint. This sets model.finetuning_type="full" and updates the language backbone, vision backbone and Mixer; "lora" selects adapter training. Set the model and data paths in both stages consistently.

python -m valen.train --config configs/train/qwen/sft_shared_state_full_joint.json \
  --initialize output/qwen-shared-state/warmup/latest

For multiple GPUs, use VALEN_GPUS=8 bash scripts/train/launch.sh <config> [--initialize <checkpoint>]. See the training guide for RLCD rewards and checkpoint recovery.

Contributions to Valen are welcome. Open an issue to report a problem, share a use case or discuss experimental results. Submit a pull request to improve the code or documentation, contribute training data or add evaluation tasks. Active contributors can become part of Valen's core development team.

Scan the QR code below to join the Valen WeChat group, discuss the project and share your experiments.

The code is released under Apache 2.0. Base models and source datasets retain their respective licenses.

Built on Qwen3.5, with the decision interface inspired by TypeSafe's Jev.