@AndreasKunar thanks for giving it a shot. I'm still tinkering to find the best combination, but you might want to try commenting out that check and try setting something like $env:CC="clang"; $env:CXX="clang++" to see if Go can pick up clang instead of gcc.

@AndreasKunar thanks for giving it a shot. I'm still tinkering to find the best combination, but you might want to try commenting out that check and try setting something like $env:CC="clang"; $env:CXX="clang++" to see if Go can pick up clang instead of gcc.

Sorry, does not work for me. Tested in powershell developer prompt and via set-commands in developer command prompt.

In line 189 in lllm\generate\gen_windows.ps1 it checks for gcc with get-command gcc and fails. It would also fail with the following line get-command mingw32-make. mingw32 and gcc are not (yet?) available for WoA, its clang LLVM.

@AndreasKunar if you install the gcc-compat MSYS2 package then you don't have to do anything (gcc becomes an alias to clang, and so on). Did you try building with these instructions? #5268 (See https://github.com/hmartinez82/ollama/blob/win_arm64_docs/docs/development.md#windows-on-arm-arm64)

I even get the installer working (the installer itself will still be non ARM64, but all the binaries inside it will be).

@AndreasKunar I'm waiting for Go to be able to detect the presence of IMM8 support by the CPU at runtime (this should be in the next release of Go https://go-review.googlesource.com/c/sys/+/595678). Once that's done, I want to create a new PR here that will have two builds of the arm64 llama.cpp binary (one armv8.2 and one armv8.7) that can be chosen at runtime just like cpu, cpu_avx, and cpu_avx2 are in x64 builds.

@AndreasKunar if you install the gcc-compat MSYS2 package then you don't have to do anything (gcc becomes an alias to clang, and so on). Did you try building with these instructions? #5268 (See https://github.com/hmartinez82/ollama/blob/win_arm64_docs/docs/development.md#windows-on-arm-arm64)

I even get the installer working (the installer itself will still be non ARM64, but all the binaries inside it will be).

Yes, I build natively on Windows on ARM (WoA), and with nearly all arm64 toolsets. But no, like with the recommendations for building llama.cpp (which I understand ollama uses/compiles), I use Visuals Studio 2022 and its included clang (LLVM), CMake installation as a fully released / supported compiler for WoA (see the build recommendations for llama.cpp on WoA), not the preliminary MSYS2 arm64 support.

So my bad, sorry! I will try and build it with these instructions, but see if I still can use my llama.cpp tooling (e.g. adding aliases to my installed clang) with just adding go.

@AndreasKunar I'm waiting for Go to be able to detect the presence of IMM8 support by the CPU at runtime (this should be in the next release of Go https://go-review.googlesource.com/c/sys/+/595678). Once that's done, I want to create a new PR here that will have two builds of the arm64 llama.cpp binary (one armv8.2 and one armv8.7) that can be chosen at runtime just like cpu, cpu_avx, and cpu_avx2 are in x64 builds.

Cool, thanks a lot!!! For Snapdragon X, llama.cpp's new Q4_0_4_4/Q4_0_4_8 quantization provides an up to 2-3x acceleration (for the compute-bound prompt-processing tasks, token-generation is mostly memory-bandwidth constrained).

@AndreasKunar #6379

No runtime dynamic detection yet of course. But that's the blunt of the changes.

@dhiltgen Can you try these steps ? https://github.com/hmartinez82/ollama/blob/win_arm64_docs/docs/development.md#windows-on-arm-arm64
With those steps, I can build an ARM64 installer without any changes to build_windows.ps1

I just realized that I repeated a bunch of things you were already doing here @dhiltgen :(

@dhiltgen I'm going ahead and close #6379 since your PR is much more comprehensive (it includes installer, etc).