Local high model.
Focus on user local device to run one model.
- Fork from llama.cpp
- Trim some old model
- Use third party library
- Just support one model to run
- Merge some code from llama.cpp by time
cmake -B build -DGGML_CUDA=OFF
cmake --build build --config Release -j 8
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j 8
cmake -B build -DGGML_CUDA=OFF -DGGML_METAL=ON
cmake --build build --config Release -j 8
./bin/llm_cli --model /modelpath/model.gguf --log_level info