Skip to content

docs: reorganize inference gateway guides - #716

Open
danehans wants to merge 10 commits into
kserve:mainfrom
danehans:reorganize-inference-gateway-docs
Open

danehans wants to merge 10 commits into
kserve:mainfrom
danehans:reorganize-inference-gateway-docs

Conversation

@danehans

@danehans danehans commented Jul 29, 2026

Copy link
Copy Markdown

Fixes: #714

Proposed Changes

  • Reorganize inference gateway integrations for agentgateway and Envoy AI Gateway.
  • Replace the legacy InferenceService AI Gateway tutorial with an LLMInferenceService gateway chooser.
  • Migrate examples to GAIE v1 and the llm-d Router endpoint-picker artifacts.
  • Document standard InferencePool routing and the optional AgentgatewayBackend policy wrapper.
  • Add the KServe and agentgateway installation order for transitional GAIE CRDs.

Validation

  • npm run build
  • Verified KServe v0.20.0 pins GAIE v1.5.0 and llm-d Router v0.9.0.
  • Kind with the v0.20 release-candidate baseline: llm-d Router with agentgateway and KServe-managed InferencePool returned HTTP 200.
  • Kind with the v0.20 release-candidate baseline: AgentgatewayBackend returned HTTP 200 and token rate limiting returned HTTP 429.

@netlify

netlify Bot commented Jul 29, 2026

Copy link
Copy Markdown

Deploy Preview for elastic-nobel-0aef7a ready!

Name Link
🔨 Latest commit dd523ea
🔍 Latest deploy log https://app.netlify.com/projects/elastic-nobel-0aef7a/deploys/6a8735cd1a15470008439113
😎 Deploy Preview https://deploy-preview-716--elastic-nobel-0aef7a.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@danehans
danehans marked this pull request as ready for review July 29, 2026 22:01
@danehans
danehans marked this pull request as draft July 29, 2026 22:08
@danehans
danehans marked this pull request as ready for review July 30, 2026 19:53
Fixes: kserve#714
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Fixes: kserve#714
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Fixes: kserve#714
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Fixes: kserve#714
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Fixes: kserve#714
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Fixes: kserve#714
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Update GAIE v1 resources, llm-d Router artifacts, and the tested agentgateway and KServe installation order.

Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Fixes: kserve#714
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Fixes: kserve#714
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@danehans
danehans force-pushed the reorganize-inference-gateway-docs branch from 3d35630 to dd523ea Compare August 20, 2026 17:13

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reorganize LLMInferenceService inference gateway documentation

1 participant