Benchmark Intel B580 on Raspberry Pi 5 #26
Description
See: geerlingguy/raspberry-pi-pcie-devices#695
Idle power consumption with desktop active is about 16.8W.
Qwen2.5 14b
$ ./build/bin/llama-bench -m models/Qwen2.5-14B-Instruct-Q4_K_M.gguf -n 128 -p 512,4096 -pg 4096,128 -ngl 99 -r 2
TODO
TODO RESULT
The full system was using about TODOW during the run.
gpt-oss 20b
$ ./build/bin/llama-bench -m models/gpt-oss-20b-F16.gguf -n 128 -p 512,4096 -pg 4096,128 -ngl 99 -r 2
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Intel(R) Arc(tm) B580 Graphics (BMG G21) (Intel open-source Mesa driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 131072 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
ggml_vulkan: Device memory allocation of size 901780992 failed.
ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory
main: error: failed to load model 'models/gpt-oss-20b-F16.gguf'
Llama 3.2:3b
$ ./build/bin/llama-bench -m models/Llama-3.2-3B-Instruct-Q4_K_M.gguf -n 128 -p 512,4096 -pg 4096,128 -ngl 99 -r 2
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Intel(R) Arc(tm) B580 Graphics (BMG G21) (Intel open-source Mesa driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 131072 | int dot: 1 | matrix cores: KHR_coopmat
build: d2ee056e (6713)
| model | size | params | backend | ngl | test | t/s |
|---|---|---|---|---|---|---|
| llama 3B Q4_K - Medium | 1.87 GiB | 3.21 B | Vulkan | 99 | pp512 | 909.56 ± 0.86 |
| llama 3B Q4_K - Medium | 1.87 GiB | 3.21 B | Vulkan | 99 | pp4096 | 801.84 ± 1.13 |
| llama 3B Q4_K - Medium | 1.87 GiB | 3.21 B | Vulkan | 99 | tg128 | 47.38 ± 1.27 |
| llama 3B Q4_K - Medium | 1.87 GiB | 3.21 B | Vulkan | 99 | pp4096+tg128 | 499.66 ± 17.60 |
The full system was using about 146W during the run.