Skip to content

TT-18010: Normalize MCP protocol context - #8594

Closed
andrei-tyk wants to merge 6 commits into
masterfrom
TT-18010-mcp-protocol-context
Closed

TT-18010: Normalize MCP protocol context#8594
andrei-tyk wants to merge 6 commits into
masterfrom
TT-18010-mcp-protocol-context

Conversation

@andrei-tyk

@andrei-tyk andrei-tyk commented Aug 18, 2026 •
edited by atlassian Bot
Loading

Copy link
Copy Markdown
Contributor

Description

Introduces a single bounded MCP ingress envelope/protocol context and exports effective/declared/source protocol fields through analytics. Dashboard pins the Pump prerequisite, covers old/new record decoding, and adds black-box analytics cases.

Related Issue

Motivation and Context

Routing, policy, errors, filtering, handler selection, and analytics need one agreed protocol interpretation instead of reparsing request bodies independently.

How This Has Been Tested

  • GOCACHE=/private/tmp/tt18006-go-cache go test ./internal/mcp/... ./internal/jsonrpc/errors -count=1 — pass on the complete Gateway stack.
  • GOCACHE=/private/tmp/tt18006-pump-gocache go test ./analytics ./serializer -count=1 — pass, covering JSON/BSON and protobuf/msgpack compatibility.
  • GOCACHE=/private/tmp/tt18006-dashboard-gocache go test ./dashboard -run 'TestMCP(TrustedOriginsManagementContract|ProtocolContextAnalyticsJSONCompatibility)' -count=1 — pass.
  • python3 -m compileall -q tests/api/mcp_client.py tests/api/tests/mcp — pass.
  • The documented .venv/bin/pytest -c pytest_local.ini tests/mcp/ ... analytics gate was not run because the live release environment is unavailable.

Types of changes

  • New feature (non-breaking change which adds functionality)
  • Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

  • Pump pin is intentionally temporary until the Pump prerequisite is released.
  • Gateway/Dashboard module graphs were tidied against the exact draft commits.
  • Replace Pump pseudo-version with the released version before readiness.

@probelabs

probelabs Bot commented Aug 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

This PR introduces a centralized parsing mechanism for the Model Context Protocol (MCP) to ensure a single, consistent interpretation of incoming requests throughout the gateway's processing pipeline.

Previously, different components like routing, policy enforcement, and analytics would independently parse the MCP request body. This led to duplicated effort and potential inconsistencies. This change refactors the JSONRPCMiddleware to parse the request body only once upon ingress. The result, a normalized ProtocolContext, is then stored in the http.Request context.

This new ProtocolContext contains the parsed request envelope, the raw body, and critically, a normalized protocol version derived from headers and body content. Downstream components, such as the REST-as-MCP policy middleware and the analytics handler, now retrieve this pre-parsed context instead of performing their own parsing. Additionally, the normalized protocol version details (EffectiveProtocolVersion, DeclaredProtocolVersion, ProtocolVersionSource) are now exported in analytics records, which required updating the tyk-pump dependency.

Files Changed Analysis

The changes are centered around the gateway's MCP handling:

  • New Context Logic (internal/mcp/protocol_context.go, internal/httpctx/mcp.go): Introduces the core ProtocolContext struct and the normalization logic for determining the MCP protocol version from various sources (headers, body, session). A new httpctx helper provides safe access to this context.
  • Middleware Refactoring (gateway/mw_jsonrpc.go): The main JSON-RPC middleware now performs a single, bounded read of the request body, creates the ProtocolContext, and stores it. The logic for REST-as-MCP adapters (gateway/mw_jsonrpc_rest_as_mcp_policy.go) was simplified to use this shared context, removing its own parsing code.
  • Analytics Enrichment (gateway/handler_success.go): The success handler now reads the protocol version from the ProtocolContext and adds it to the analytics record.
  • Context Key (ctx/ctx.go): A new key, MCPProtocolContext, is added to the shared context definitions.
  • Dependencies (go.mod, go.sum): The tyk-pump dependency is updated to a newer version capable of handling the enriched analytics records.
  • Testing: New unit tests were added for the protocol context normalization logic, and existing middleware tests were updated to reflect the single-parse behavior.

Architecture & Impact Assessment

  • What this PR accomplishes: It refactors MCP request handling to be more efficient and consistent by implementing a "parse-once, use-many" pattern. This eliminates redundant processing and establishes a single source of truth for the request's protocol version and content.

  • Key technical changes introduced:

    1. Centralized Ingress Parsing: The JSONRPCMiddleware is now the sole owner of MCP request body parsing.
    2. Request-Scoped Protocol Context: A new mcp.ProtocolContext object is attached to each request, carrying the parsed envelope and normalized metadata.
    3. Protocol Version Normalization: A new algorithm determines the effective MCP version by evaluating headers, body fields, and session state, and handles potential conflicts.
    4. Enriched Analytics: Analytics data for MCP requests is now augmented with detailed protocol version information.

  • Affected system components:

    • Gateway: Core request processing for MCP APIs is fundamentally changed.
    • Analytics Pipeline: The tyk-pump component needs the corresponding update (included in this PR via go.mod) to process the new analytics fields. The tyk-analytics dashboard will also require changes to visualize this new data.

Request Flow Transformation

This diagram illustrates how the change centralizes request parsing.

Before: Multiple components read and parsed the request body independently.

sequenceDiagram
    participant Client
    participant JSONRPCMiddleware
    participant Other Middlewares
    participant SuccessHandler

    Client->>JSONRPCMiddleware: POST /mcp request
    JSONRPCMiddleware->>JSONRPCMiddleware: Read & Parse Body for routing
    JSONRPCMiddleware->>Other Middlewares: Forward request
    Other Middlewares->>Other Middlewares: Read & Parse Body again for policy
    Other Middlewares->>SuccessHandler: Forward request
    SuccessHandler->>SuccessHandler: Read & Parse Body again for analytics
Loading

After: The body is parsed once, and the result is shared via the request context.

sequenceDiagram
    participant Client
    participant JSONRPCMiddleware
    participant Other Middlewares
    participant SuccessHandler

    Client->>JSONRPCMiddleware: POST /mcp request
    JSONRPCMiddleware->>JSONRPCMiddleware: Read & Parse Body ONCE
    JSONRPCMiddleware->>JSONRPCMiddleware: Store ProtocolContext in http.Request
    JSONRPCMiddleware->>Other Middlewares: Forward request
    Other Middlewares->>Other Middlewares: Get ProtocolContext from http.Request
    Other Middlewares->>SuccessHandler: Forward request
    SuccessHandler->>SuccessHandler: Get ProtocolContext from http.Request
Loading

Scope Discovery & Context Expansion

This PR is part of a larger, cross-repository initiative. The changes to analytics data in the Gateway are only useful when the downstream components are prepared for them. The PR description and go.mod changes confirm this by referencing corresponding branches and updated versions for tyk-pump and tyk-analytics (Dashboard).

The introduction of the ProtocolContext is a significant architectural improvement. Future development on MCP-related features should leverage this centralized context rather than introducing new parsing logic. The change primarily impacts the gateway and its interaction with the analytics stack.

Metadata
  • Review Effort: 3 / 5
  • Primary Label: feature

Powered by Visor from Probelabs

Last updated: 2026-08-18T08:58:37.507Z | Triggered by: pr_updated | Commit: e518d59

💡 TIP: You can chat with Visor using /visor ask <your question>

@probelabs

probelabs Bot commented Aug 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Security Issues (1)

Severity Location Issue
🟡 Warning internal/mcp/protocol_context.go:42
The `ProtocolContext` struct, which is stored in the request context for the duration of the request, contains the raw request body in the `RawBody` field. While this field is not used in a way that exposes data in the current pull request, its presence in a widely accessible context object creates a risk of future sensitive data exposure. A developer might inadvertently log the entire `ProtocolContext` object for debugging, which could leak sensitive information from the request payload into logs.
💡 SuggestionTo mitigate the risk of accidental data leakage, implement a `String()` or `MarshalJSON()` method for the `ProtocolContext` struct that redacts the `RawBody` field. This prevents the raw body from being included when the struct is formatted for logging. Alternatively, add a prominent comment to the struct definition warning developers against logging this field or the containing struct directly.

Security Issues (1)

Severity Location Issue
🟡 Warning internal/mcp/protocol_context.go:42
The `ProtocolContext` struct, which is stored in the request context for the duration of the request, contains the raw request body in the `RawBody` field. While this field is not used in a way that exposes data in the current pull request, its presence in a widely accessible context object creates a risk of future sensitive data exposure. A developer might inadvertently log the entire `ProtocolContext` object for debugging, which could leak sensitive information from the request payload into logs.
💡 SuggestionTo mitigate the risk of accidental data leakage, implement a `String()` or `MarshalJSON()` method for the `ProtocolContext` struct that redacts the `RawBody` field. This prevents the raw body from being included when the struct is formatted for logging. Alternatively, add a prominent comment to the struct definition warning developers against logging this field or the containing struct directly.
\n\n ### Architecture Issues (3)
Severity Location Issue
🟡 Warning gateway/mw_jsonrpc.go:102-104
The logic for setting the protocol context on error is duplicated across three separate error-handling blocks within the `readAndParseJSONRPC` function. This creates redundant code that can be consolidated.
💡 SuggestionRefactor the function to use a `defer` statement. This will ensure the protocol context is always set, regardless of whether parsing succeeds or fails, by centralizing the context creation and setting logic. This approach simplifies the function's control flow and removes the repeated code blocks.
🟡 Warning gateway/mw_jsonrpc.go:185-191
The `ProcessRequest` function contains a conditional check for `m.Spec.IsSyntheticMCPAdapter()` that appears to be redundant. The main request validation and parsing logic is executed for all JSON-RPC requests, and the synthetic adapter path is handled within that flow. The initial check seems to be a remnant of a previous implementation and adds unnecessary complexity to the control flow.
💡 SuggestionRemove the initial check for `IsSyntheticMCPAdapter()` at the beginning of the `ProcessRequest` function. The subsequent logic correctly handles both standard and synthetic MCP requests after parsing the request body and setting the protocol context.
🟡 Warning internal/mcp/protocol_context.go:117-126
The logic for extracting and unmarshaling a protocol version from a JSON raw message is repeated for both `protocolVersion` and `_meta` fields within `extractBodyProtocolVersion`. This creates duplicated code for parsing the version string from different keys.
💡 SuggestionCreate a small helper function or a closure within `extractBodyProtocolVersion` that takes a `json.RawMessage` as input and returns the unmarshaled version string and a boolean indicating success or failure. This helper can then be called for both `params["protocolVersion"]` and `metadata[MetaKeyProtocolVersion]`, eliminating the repeated unmarshaling and validation logic.

Performance Issues (1)

Severity Location Issue
🟡 Warning internal/mcp/protocol_context.go:64
The `NewProtocolContext` function creates a full copy of the raw request body for each request via `append([]byte(nil), rawBody...)`. While this provides safety against downstream modifications of the byte slice, it introduces a memory allocation and copy overhead for every MCP request. With a body limit of 1MB, this can contribute to significant memory usage and GC pressure under high load.
💡 SuggestionThe `rawBody` byte slice originates from `io.ReadAll` within `readAndParseJSONRPC` and is not otherwise exposed in a way that encourages modification. Standard library consumers of `http.Request.Body` are expected to treat read bytes as immutable. Consider removing this defensive copy and treating the slice as read-only throughout its lifecycle to reduce per-request allocations. If the copy is deemed essential for safety, add a comment explaining the reason.

Powered by Visor from Probelabs

Last updated: 2026-08-18T08:57:59.983Z | Triggered by: pr_updated | Commit: e518d59

💡 TIP: You can chat with Visor using /visor ask <your question>

@andrei-tyk andrei-tyk mentioned this pull request Aug 18, 2026
Closed
4 tasks
@andrei-tyk
andrei-tyk force-pushed the TT-18010-mcp-protocol-context branch from becc05b to e518d59 Compare August 18, 2026 08:56
@andrei-tyk

andrei-tyk commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by the current v2 implementation in #8678.

@andrei-tyk andrei-tyk closed this Sep 16, 2026
@andrei-tyk andrei-tyk reopened this Sep 16, 2026
@andrei-tyk
andrei-tyk changed the base branch from TT-18030-mcp-origin-validation to master September 16, 2026 22:04
@andrei-tyk andrei-tyk closed this Sep 16, 2026
@sentinelone-cnapp-eu1

sentinelone-cnapp-eu1 Bot commented Sep 16, 2026

Copy link
Copy Markdown

SentinelOne CNS Hardcoded Secret Detector
✅ Congratulations, your code is safe

SentinelOne CNS is a cloud-agnostic, agentless CSPM & CWPP solution that continuously detects and prevents vulnerabilities that have the highest probability of being exploited in Azure, AWS, Google Cloud, and Kubernetes.

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

🎯 Recommended Merge Targets

Based on JIRA ticket TT-18010: Detect and record protocol version

Fix Version: Tyk 5.16.0

⚠️ Warning: Expected release branches not found in repository

Required:

  • master - No matching release branches found. Fix will be included in future releases.

📋 Workflow

  1. Merge this PR to master first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@andrei-tyk