Repository navigation
How do I specify the path to my local llama.cpp in Ubuntu? #141
Description
Activity
@SergProduction You could use settings launch_completion, launch_chat, lanuch_embeddings and launch_tools. For example in launch_chat you could enter:
cd /path/to/you/bin && llama-server -hf ggml-org/Qwen2.5-Coder-1.5B-Instruct-Q8_0-GGUF -ngl 99 -ub 1024 -b 1024 --ctx-size 0 --cache-reuse 256 -np 2 --port 8011(or -m instead of -hf if you want to specify the location of the local model)
After that you could start your command/model by selecting "Use settings..." from the chat models list:
I don't want to set a global PATH in the system, because I'm experimenting with assemblies and making several instances, they are in different directories and I want to switch between them, I also have several local models with which I experiment, I would like to specify the path to bin and models in the extension settings