-
|
Hi, does the Immich machine learning module utilize AVX-512 and AVX-VNNI? The OpenVINO framework seems to be able to make use of these instruction sets, but Immich only mentions GPU support in this context. Is CPU acceleration by use of specialized instruction sets something that works out of the box, or does Immich use OpenVINO in a way that only makes use of GPUs? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
|
cc @mertalev |
Beta Was this translation helpful? Give feedback.
-
|
Hi! ONNX Runtime can use AVX-512/VNNI, yes. We run models at full precision, though. I believe quantized models take more advantage of these instructions. |
Beta Was this translation helpful? Give feedback.
Hi! ONNX Runtime can use AVX-512/VNNI, yes. We run models at full precision, though. I believe quantized models take more advantage of these instructions.