Home for the Development of MLX Vulkan backend
CI benchmark history from AMD Radeon 8060S (Strix Halo). Detailed data is in benchmarks/results.csv.
| Model | Bits | Prompt TPS | Generation TPS | Peak memory (GB) | mlx-vulkan | mlx | Run |
|---|---|---|---|---|---|---|---|
| mlx-community/Qwen3-0.6B-8bit | 8bit | 1360.581 | 87.609 | 2.056 | c0c3da7 | 5d618c8 | run |
| mlx-community/Qwen3-0.6B-bf16 | bf16 | 2410.127 | 65.748 | 2.614 | c0c3da7 | 5d618c8 | run |
| mlx-community/Qwen3.6-35B-A3B-8bit | 8bit | 121.011 | 21.135 | 40.350 | c0c3da7 | 5d618c8 | run |
Serial generation smoke tests validate that each model produces coherent output on Vulkan.
| Model | Output | Coherent | Peak memory (GB) | Sample | Error |
|---|---|---|---|---|---|
| mlx-community/Qwen3-0.6B-bf16 | pass | pass | 1.148 | Okay, the user wants a concise sentence about why Vulkan acceleration is useful. Let... | |
| mlx-community/Qwen3-0.6B-8bit | pass | pass | 1.032 | Okay, the user wants a concise sentence about why Vulkan acceleration is useful. Let... | |
| LiquidAI/LFM2.5-1.2B-Instruct-MLX-8bit | pass | pass | 1.396 | Vulkan acceleration enhances performance by enabling efficient parallel processing and reduci... | |
| mlx-community/Qwen3.5-2B-bf16 | pass | pass | 4.529 | Thinking Process: 1. Analyze the Request: * Task: Write one concise sentence. * Topic: Wh... | |
| mlx-community/gemma-4-e2b-it-bf16 | pass | pass | 10.005 | <|channel>thought 1. Analyze the Request: The user wants a concise sentence explaining... | |
| mlx-community/gemma-4-e4b-it-4bit | pass | pass | 5.19 | <|channel>thought 1. Analyze the request: The user wants one concise sentence explainin... | |
| mlx-community/gemma-4-26b-a4b-it-4bit | pass | pass | 14.092 | <|channel>thought * Topic: Why Vulkan acceleration is useful. * Constraint: One concise sente... | |
| mlx-community/Qwen3.6-35B-A3B-8bit | pass | pass | 35.819 | Here's a thinking process: 1. Analyze User Input: - Topic: Vulkan acceleration - **Re... | |
| mlx-community/gpt-oss-20b-MXFP4-Q8 | pass | pass | 13.689 | <|channel|>analysis<|message|>We need to write one concise sentence about why Vulkan accelera... | |
| mlx-community/Qwen3.6-27B-8bit | pass | pass | 28.461 | Here's a thinking process: 1. Analyze User Input: - Topic: Vulkan acceleration - **Re... |