Skip to content

Warn and ignore typical_p on API requests instead of rejecting them - #18551

Open
SIDDARTHAREDDY8 wants to merge 1 commit into
ollama:mainfrom
SIDDARTHAREDDY8:bhai-blitz-9-ollama-1
Open

SIDDARTHAREDDY8 wants to merge 1 commit into
ollama:mainfrom
SIDDARTHAREDDY8:bhai-blitz-9-ollama-1

Conversation

@SIDDARTHAREDDY8

Copy link
Copy Markdown

Fixes #18542.

PR #18448 made scheduleRunner reject any generate/chat request that carries typical_p with a 400 ("typical_p is no longer supported"). That breaks existing clients that cannot omit the parameter: e.g. SillyTavern always sends typical_p (set to the no-op value 1.0), so upgrading to 0.34.1 makes all its generations fail even though the client is not trying to use typical-p sampling.

Change: scheduleRunner now logs a warning and drops typical_p from the request options instead of failing the request. Model-level typical_p values stored in existing models are still honored through the model options path, and the ollama create / PARAMETER typical_p rejection is unchanged (that is a deliberate user action where a hard error is appropriate).

Tests:

  • Updated TestGenerate/ignored_deprecated_option (was rejected option): a generate request with Options: {"typical_p": 0.5} now returns 200 instead of 400.
  • go vet ./server/ clean; go test ./server/ -run TestGenerate passes (including the updated subtest and the adjacent null option is unset case). Full ./server suite has one pre-existing failure (TestPullManifestRedirectPolicy) which also fails on the unmodified tree.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

typical_p is no longer supported breaks existing clients that cannot omit the parameter

1 participant