Surface PUT _inference validation failures as 4xx instead of 5xx #150084

Elasticsearch Version

All

Installed Plugins

No response

Java Version

bundled

OS Version

All

Problem Description

Summary

When a client calls PUT _inference/text_embedding/<id> with a service config that omits dimensions, the validation throw appears to escape the request flow and is logged at ERROR via ElasticsearchUncaughtExceptionHandler. The validation itself is correct. What looks off is the routing: the caller does not seem to receive a clean 400, and the ERROR level log lines trip downstream log error quality gates.

Stack trace (identical across every occurrence we observed):

java.lang.IllegalArgumentException: required [dimensions] field is missing for task_type [TEXT_EMBEDDING]
  at org.elasticsearch.server@9.5.0/org.elasticsearch.inference.MinimalServiceSettings.validateFieldPresent(MinimalServiceSettings.java:260)
  at org.elasticsearch.server@9.5.0/org.elasticsearch.inference.MinimalServiceSettings.validate(MinimalServiceSettings.java:245)
  at org.elasticsearch.server@9.5.0/org.elasticsearch.inference.MinimalServiceSettings.<init>(MinimalServiceSettings.java:141)
  at org.elasticsearch.server@9.5.0/org.elasticsearch.inference.MinimalServiceSettings.<init>(MinimalServiceSettings.java:155)
  at org.elasticsearch.inference@9.5.0/org.elasticsearch.xpack.inference.action.TransportPutInferenceModelAction.findIndicesWithIncompatibleMappings(TransportPutInferenceModelAction.java:286)
  at org.elasticsearch.inference@9.5.0/org.elasticsearch.xpack.inference.action.TransportPutInferenceModelAction.checkForExistingUsesOfInferenceId(TransportPutInferenceModelAction.java:268)
  at org.elasticsearch.inference@9.5.0/org.elasticsearch.xpack.inference.action.TransportPutInferenceModelAction.lambda$parseAndStoreModel$4(TransportPutInferenceModelAction.java:260)
  at org.elasticsearch.server@9.5.0/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:1047)

Logged via log.logger: org.elasticsearch.bootstrap.ElasticsearchUncaughtExceptionHandler with log.level: ERROR.

Cause

In x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/action/TransportPutInferenceModelAction.java, the work submitted to the utility executor at line 260 reaches findIndicesWithIncompatibleMappings (line 286), which constructs new MinimalServiceSettings(model) and triggers the validation in server/src/main/java/org/elasticsearch/inference/MinimalServiceSettings.java:260. As far as we can tell from reading the code, the resulting IllegalArgumentException is not routed to modelValidatingListener.onFailure(...) and ends up in the executor's uncaught exception handler.

Suggestion

This looks like a client side input error that should surface to the caller as a 4xx (likely 400 Bad Request, given the existing RestStatus.BAD_REQUEST already used elsewhere in the same handler), rather than escaping as an uncaught exception logged at ERROR. The same validation message can stay; the change is in how it is propagated.

Note

The 12 occurrences we observed in the 24h window ending 2026-05-28 11:21 UTC (approximately 2026-05-27 11:21 UTC to 2026-05-28 11:21 UTC) all came from one project on production canary repeatedly calling PUT _inference with the same malformed config, so there is likely a misbehaving client or integration on that project side as a secondary issue. We only noticed this surface because it tripped the check-log-error-rate quality gate of the serverless production canary promotion (threshold 0 errors), which required manual triage to confirm it was not a regression in the candidate build and effectively delayed the promotion.

Sample log entries in Kibana (Elastic internal, requires login):

https://overview.elastic-cloud.com/app/discover#/?_tab=(tabId:b792a1ba-a4a0-492f-80d1-9d2e4c33f6ab)&_g=(filters:!(),refreshInterval:(pause:!t,value:60000),time:(from:now-24h%2Fh,to:now))&_a=(breakdownField:log.level,columns:!(),dataSource:(dataViewId:'logging-*:logs-*',type:dataView),filters:!(),hideChart:!f,interval:auto,query:(language:kuery,query:'log.level%20:%20%22ERROR%22%0A%20%20%20%20and%20kubernetes.annotations.elastic_co%2Fversion%20:%20%22git-f4e0cfa78136%22%0A%20%20%20%20and%20data_stream.dataset%20:%20(%22elasticsearch.log%22%20or%20%22elasticsearch.server%22)'),sort:!(!('@timestamp',desc)))

The URL uses a relative time range (now-24h/h to now), so it will show whatever is in the last 24 hours at the time of clicking.

Equivalent filters if entering KQL by hand. Data view: logging-*:logs-*. Time range: last 24h (or set absolute around the window in this issue if events have rolled out of retention). Query:

log.level : "ERROR"
  and kubernetes.annotations.elastic_co/version : "git-f4e0cfa78136"
  and data_stream.dataset : ("elasticsearch.log" or "elasticsearch.server")

To narrow to just the dimensions errors, add:

and message : "field is missing for task_type"

Steps to Reproduce

1. Have an index referencing an inference id via semantic_text.
2. PUT _inference/text_embedding/<that-id> with dimensions omitted.
3. Observe an ERROR log via ElasticsearchUncaughtExceptionHandler instead of a clean 4xx.

Logs (if relevant)

No response