v0.8.2
Expected Behavior
In previous versions of Jan AI, users could configure global inference and sampling parameters (such as temperature, top_p, min_p, and presence_penalty) directly within the main GUI settings or sidebar layout. These settings applied globally across the local inference engine, ensuring that external API calls—such as connecting Jan AI's OpenAI-compliant server endpoint (/v1/chat/completions) to external coding agents (e.g., Continue, Roo Code, Aider)—respected these custom configurations.
Actual Behavior
Following recent UI overhauls and migration to the newer llama.cpp / unified inference server layer, sampling parameters have been moved exclusively into the Chat UI (via the chat input popover/sliders menu). Because there is no longer a global or model-level configuration panel for these samplers in the main settings GUI, the local API server falls back to rigid defaults. This leaves headless/API-driven workflows entirely broken or forced into manual JSON/YAML editing workarounds within AppData.
Steps to Reproduce
Open Jan AI (latest release).
Attempt to locate global top_p or temperature controls outside an active UI chat thread (e.g., in the main Settings gear layout or the specific model management tab).
Connect an external IDE coding agent or custom script to http://localhost:1337/v1.
Observe that the model ignores expected coding parameters unless the calling agent explicitly overrides the payload, as Jan's GUI offers no way to set API default parameters natively anymore.
Environment Context
OS: Windows 10/11 x64
App Version: Latest architecture release
Processor Extension: AVX-512 / AVX2
Inference Engine: llama.cpp / cortex
Additional Notes / Suggested Fix
While moving sampling options into a popover next to the chat bar works well for the built-in Chat interface, it strips vital utility from power users who treat Jan AI primarily as a local API server pipeline.
Proposed Solution: Please expose an "API Defaults" or advanced sampling parameter subsection inside the model profile settings or under the App/Provider settings panel, so headless API configurations can be managed seamlessly from the GUI again.
v0.8.2
Expected Behavior
In previous versions of Jan AI, users could configure global inference and sampling parameters (such as temperature, top_p, min_p, and presence_penalty) directly within the main GUI settings or sidebar layout. These settings applied globally across the local inference engine, ensuring that external API calls—such as connecting Jan AI's OpenAI-compliant server endpoint (/v1/chat/completions) to external coding agents (e.g., Continue, Roo Code, Aider)—respected these custom configurations.
Actual Behavior
Following recent UI overhauls and migration to the newer llama.cpp / unified inference server layer, sampling parameters have been moved exclusively into the Chat UI (via the chat input popover/sliders menu). Because there is no longer a global or model-level configuration panel for these samplers in the main settings GUI, the local API server falls back to rigid defaults. This leaves headless/API-driven workflows entirely broken or forced into manual JSON/YAML editing workarounds within AppData.
Steps to Reproduce
Open Jan AI (latest release).
Attempt to locate global top_p or temperature controls outside an active UI chat thread (e.g., in the main Settings gear layout or the specific model management tab).
Connect an external IDE coding agent or custom script to http://localhost:1337/v1.
Observe that the model ignores expected coding parameters unless the calling agent explicitly overrides the payload, as Jan's GUI offers no way to set API default parameters natively anymore.
Environment Context
OS: Windows 10/11 x64
App Version: Latest architecture release
Processor Extension: AVX-512 / AVX2
Inference Engine: llama.cpp / cortex
Additional Notes / Suggested Fix
While moving sampling options into a popover next to the chat bar works well for the built-in Chat interface, it strips vital utility from power users who treat Jan AI primarily as a local API server pipeline.
Proposed Solution: Please expose an "API Defaults" or advanced sampling parameter subsection inside the model profile settings or under the App/Provider settings panel, so headless API configurations can be managed seamlessly from the GUI again.