Repository navigation
[inference] Sanitize tool schemas for Google Vertex AI inference endpoints - #284810
Conversation
| @@ -0,0 +1,122 @@ | |||
| /* | |||
There was a problem hiding this comment.
Comments in this file are verbose, but considering what it does, I think it's worth keeping them to be explicitly clear about why these functions are needed. Will remove if anyone thinks otherwise.
There was a problem hiding this comment.
One concrete correctness finding on the Vertex schema allowlist (oneOf is not a supported Vertex proto field). Left inline. Otherwise the approach (allowlist rebuild + provider gating) looks sound.
Generated by Claude Reviewer for #284810 · 127.4 AIC · ⌖ 16.4 AIC · ⊞ 4.6K
| 'pattern', | ||
| 'example', | ||
| 'anyOf', | ||
| 'oneOf', |
There was a problem hiding this comment.
oneOf is not a field in Vertex's Schema proto — that message defines any_of (field 11) but no one_of/oneOf. Since the PR's stated failure mechanism is that Vertex rejects unknown proto fields with a 400, keeping oneOf in the allowlist means discriminated-union tool schemas still fail.
zod v4 emits oneOf for z.discriminatedUnion (as the operations fixture in sanitize_tool_schemas_for_vertex.test.ts shows), and the recursion below preserves it as-is, so the sanitized output still contains oneOf. The test asserts this and therefore locks in the broken behavior — discriminated-union tools will 400 against a real Vertex endpoint, the same class of bug this PR is fixing. The live end-to-end check ("default agent works") likely didn't exercise a discriminated-union tool schema, so this path went unverified.
Suggested fix: drop 'oneOf' from the allowlist and convert oneOf → anyOf during recursion (this is what the referenced LiteLLM / genai-SDK implementations do), then update the test expectation accordingly.
There was a problem hiding this comment.
Interestingly, I did test oneOf and it worked!
The parser steps into oneOf successfully and the failure is with const (which I have fixed in this PR).
But I'll look into this further, I'd prefer that we conform to the exact documented schema so the bot's comment is valid.
💛 Build succeeded, but was flaky
Failed CI Steps
Metrics [docs]
Test Failures
cc @chrisbmar |
| const sanitizedTools = | ||
| provider === InferenceEndpointProvider.GoogleVertexAI | ||
| ? sanitizeToolSchemasForVertex(tools) | ||
| : tools; |
There was a problem hiding this comment.
I would move that inside createEndpointRequest, better isolation of concerns ihmo
There was a problem hiding this comment.
Simple condition, if it ever expands to more providers, definitely makes sense, gonna merge now to unblock.
| /** | ||
| * Rebuilds a tool schema keeping only the fields Vertex AI accepts, so that | ||
| * JSON Schema keywords zod v4 (or user-provided tool schemas) emit never reach | ||
| * Vertex's parser. Conversions preserve the intent of dropped fields: | ||
| * `const: 'x'` becomes `enum: ['x']`, `oneOf` branches become `anyOf`, and a | ||
| * `null` branch in `anyOf` (zod's representation of nullable values) becomes | ||
| * `nullable: true`. | ||
| */ | ||
| const toVertexSchema = <T extends ToolSchemaType>(schemaPart: T): T => { |
There was a problem hiding this comment.
Yeah I really hate that we have to do this, it's crazy that they didn't improve this in 2 years, but I don't see an alternative, so...
|
@chrisbmar have you tested the behavior with the (legacy) Kibana gemini connector? I suspect we may want/need to do the same thing here too? |
Yes, see PR description:
|
|
Starting backport for target branches: 9.5 |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…e endpoints (#284810) (#285146) # Backport This will backport the following commits from `main` to `9.5`: - [[inference] Sanitize tool schemas for Google Vertex AI inference endpoints (#284810)](#284810) <!--- Backport version: 11.0.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Chris","email":"50221462+chrisbmar@users.noreply.github.com"},"sourceCommit":{"committedDate":"2026-08-14T08:28:54Z","message":"[inference] Sanitize tool schemas for Google Vertex AI inference endpoints (#284810)","sha":"9ab5bdc4277d70b8e650966afee40fdcf3451827","branchLabelMapping":{"^v9.6.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","backport:version","v9.5.0","v9.6.0"],"title":"[inference] Sanitize tool schemas for Google Vertex AI inference endpoints","number":284810,"url":"https://github.com/elastic/kibana/pull/284810","mergeCommit":{"message":"[inference] Sanitize tool schemas for Google Vertex AI inference endpoints (#284810)","sha":"9ab5bdc4277d70b8e650966afee40fdcf3451827"}},"sourceBranch":"main","suggestedTargetBranches":["9.5"],"targetPullRequestStates":[{"branch":"9.5","label":"v9.5.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.6.0","branchLabelMappingKey":"^v9.6.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/284810","number":284810,"mergeCommit":{"message":"[inference] Sanitize tool schemas for Google Vertex AI inference endpoints (#284810)","sha":"9ab5bdc4277d70b8e650966afee40fdcf3451827"}}]}] BACKPORT-->
Summary
Closes #284647
Agent Builder (and any
chatCompleteconsumer) fails with a 400 when using a Google Vertex AIchat_completioninference endpoint: tool schemas produced by zod v4'stoJSONSchemacontain JSON Schema keywords (propertyNames,exclusiveMinimum/exclusiveMaximum,const, nested inoneOf/anyOf) that Vertex's function declaration schema — a strict OpenAPI 3.0 subset — rejects as unknown proto fields.The
inference_endpointadapter (the path all inference-endpoint traffic takes since #258530) applies no tool schema transforms — this was lost when the path was added in #256071. The deprecated Gemini connector is unaffected because its adapter rebuilds schemas via an allowlist. (toolSchemaToGeminiinadapters/gemini/gemini_adapter.ts) - is the same approach this PR applies to the endpoint path, in OpenAI-format instead of the Gemini SDK's.Changes
sanitizeToolSchemasForVertex: recursively rebuilds tool schemas keeping only the fields Vertex accepts (allowlist sourced from the Vertex Schema definition — the consensus approach used by LiteLLM (filter_schema_fields), the Vercel AI SDK (convertJSONSchemaToOpenAPISchema), and Google's own genai SDK (processJsonSchema)), with two intent-preserving conversions:const: "x"→enum: ["x"]andanyOf-with-{type: "null"}→nullable: true.inference_endpoint_adapterapplies it only when the endpoint's service isgooglevertexai;callback_api.tsthreads the endpoint's service through asprovider. All other providers pass through unchanged (Bedrock endpoints verified working live without transforms).adapters/inference) is intentionally untouched.Testing
z.toJSONSchema(mirroringresolveToolSchemain@kbn/inference-langchain) and assert the offending keywords are present pre-sanitization, covering records, number bounds, literals, unions, nullable, and discriminated-union-in-array — the exact shapes observed failing live.gemini-2.5-flash): default agent works; before the fix both reproduce the customer's 400.Note: multi-round tool use against Vertex endpoints still requires the ES-side fix for tool-call arguments in this PR elastic/elasticsearch#156665.
mapStrings()rejects non-string argument values).Before
After
