Skip to content

feat: 增加了 MCP exp-app 相关标准的支持,另外增加了project mode。#77

Open
DivXPro wants to merge 52 commits into
qqqqqf-q:mainfrom
DivXPro:feat/part2aegntClient
Open

feat: 增加了 MCP exp-app 相关标准的支持,另外增加了project mode。#77
DivXPro wants to merge 52 commits into
qqqqqf-q:mainfrom
DivXPro:feat/part2aegntClient

Conversation

@DivXPro

@DivXPro DivXPro commented May 21, 2026

Copy link
Copy Markdown
Contributor

MCP 的 HTTP app 基本上支持了。支持了 inline 和 full screen 两种 display 模式(fullscreen 采用点击在side Panel展开显示)。增加了 project mode,在 pipeline 会增加从 thread 表中 写入和读取 Project centext

DivXPro added 30 commits May 19, 2026 16:18
- Add ToolAnnotations type in llm package (DestructiveHint, ReadOnlyHint,
  IdempotentHint, OpenWorldHint, Title)
- Extract annotations and _meta from SDK Tool in sdk_client ListTools
- Pass annotations through AgentToolSpec and ToolSpec
- Use annotations to refine RiskLevel and SideEffects on MCP tools
- Add 12 unit tests for convertAnnotations, mcpRiskLevel, mcpSideEffects,
  and ToolSpec.ToJSON
Worker:
- Add Resource/ResourceContent types with annotations.priority
- Add ListResources/ReadResource to Client interface
- Add ext-apps capability negotiation (io.modelcontextprotocol/ui)
- Add visibility filtering for model-invisible tools
- Extract _meta.ui.resourceUri for tool executor
- Auto-fetch MCP resource HTML on tool call, upload as artifact
- Extract CSP from resource._meta.ui.csp
- Add PartTypeResource to message content pipeline

Web:
- Add McpAppIframe component (sandboxed iframe + postMessage + theme sync)
- Add MCP App shim (__MCP_APP__) in ArtifactIframe for inline app rendering
- Add McpAppCsp/McpAppResource types + localStorage persistence
API:
- Add project_meta_context JSONB column to threads table (migration 00198)
- Add ProjectMetaContext field to Thread struct
- Pass project_meta_context to run.started event data

Worker:
- Add ProjectMetaContext field to RunContext
- Add ApplyProjectMetaContext to InputLoader
- Add ApplyProjectContext middleware (injects system_reminder)
- Auto-detect od_create_project tool result and emit project_meta_context event
- Add applyThreadProjectMetaContextEvent handler
- Add SQLite migration 00105 for project_meta_context column
- Add applyThreadProjectMetaContextEvent handler to desktop EventWriter
- Desktop sidecar now compatible with Project Context Mode
When ArtifactStore is nil (desktop mode), MCP resource HTML was incorrectly
sent as ContentAttachment to the LLM, causing Anthropic API to reject it as
'unsupported image format: text/html'. Now stored inline in resultJSON resources.
Desktop had artifact store but wasn't passing it to the MCP pool,
causing MCP ext-apps resource HTML to fall back to inline storage
instead of being uploaded as proper artifacts.
- Add extractResources() and buildMessageResourcesFromAgentEvents() in agentEventProcessing
- Add resources Map to ChatView (loading, replay, merge to MessageMeta)
- Add resources field to MessageMeta type
- Add resource persistence read/write in message-meta context
- MCP tool results with resources[] now flow through the full data pipeline
- Add ResourceUIPreview component (fetches artifact HTML, renders McpAppIframe)
- Render resources inline in MessageList after cop segments
- Mirrors development branch implementation
McpAppIframe was missing the MCP App runtime that OD HTML content
expects. Added both window.__MCP_APP__ (for script-based calls) and
importmap (for ESM imports of @modelcontextprotocol/ext-apps).
- Checkout McpAppIframe.tsx from development (uses AppBridge)
- Checkout public/mcp-ext-apps/app-with-deps.js (337KB bundled runtime)
- Install @modelcontextprotocol/sdk and @modelcontextprotocol/ext-apps deps
Desktop mode stores HTML content directly in initialData.
Skip API fetch when content is already available inline.
Previous edits introduced duplicate imports and code fragments.
Development branch already has the complete resource data pipeline
(resourcesMap, replay, mergeMeta) matching our implementation.
Port the streaming resource tracking mechanism from development:
- currentRunResourcesRef in message-meta context (SSE hot-path write)
- runResources in persistRunDataToMessage (immediate meta update)
- runResources in persistThreadRunHandoff (cross-session persistence)
- currentRunResourcesRef population from handoff + replay in ChatView
- Clears on thread change / run completion
- Updates useRunTransition TerminalRunCache type

Fixes MCP app resources not rendering for new messages - resources
now flow through streaming refs to meta immediately on run completion.
Add live resource extraction in useThreadSseEffect - mirrors development
by extracting resources from tool-result events during streaming and
pushing to currentRunResourcesRef. This is the critical missing piece:
without it, resources only flow through the replay path after run completion.
Artifact store path now also includes 'content' field so frontend
always has inline HTML available via initialData, avoiding the need
to fetch from artifact API (which may not be available in desktop mode).
- Add 'content' field to McpAppResource for inline HTML
- extractResources uses 'content' field instead of 'initialData'
- ResourceUIPreview reads from resource.content for rendering
- initialData remains for the original tool result JSON (AppBridge)
SSE event data has resources at data.result.resources, not data.resources.
Also handles the agentEventToolOutput() pre-extracted path.
…edback loop

- Add initialData field to McpAppResource so AppBridge can send tool result
  data to iframe MCP apps (was causing 'loading' state with no rendered content)
- Detect full HTML documents in McpAppIframe and inject scripts into <head>
  instead of nesting content in template (prevented malformed DOM)
- Replace getBoundingClientRect height measurement with parent-side
  ResizeObserver on iframe body to break feedback loop causing infinite growth
- Reset lastHeightRef on iframe reload to prevent cumulative height across reloads
DivXPro and others added 22 commits May 20, 2026 15:52
…ource types

- Backend: add server_id to MCP resource in executor.go (inline + artifact paths)
- Backend: implement /v1/mcp/tool-call API proxy endpoint with proper auth, config lookup, and SDK transport support
- Worker: add UIOnlyExecutors to Registration and isToolAppOnly visibility detection
- Frontend: add serverId to McpAppResource, McpAppResourceRef, and extractResources
- Frontend: implement bridge.oncalltool in McpAppIframe to call /v1/mcp/tool-call
- Frontend: add card display mode and expand-to-panel for MCP app resources
- Frontend: wire serverId through ResourceUIPreview, ResourcePreviewPanel, MessageList
- AppBridge initialization now passes hostContext.displayMode and
  availableDisplayModes to support spec-compliant mode negotiation
- Add onrequestdisplaymode handler: App can request fullscreen,
  which triggers openResourcePanel via onExpand callback
- Add onsizechange handler to support ext-apps standard size-changed
  notifications alongside existing arkloop:mcpapp:resize postMessage
- Inline mode renders with a unified card container (header + iframe)
  using --c-bg-sub background to distinguish from chat background
- Header displays MCP server name (serverId) and resource uri
- MessageList defaults all MCP resources to inline display mode

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add maxHeight and overflowY: auto to select/oneOf/multi-select
  fields to prevent long option lists from breaking layout
- Remove duplicate migration 00105 (column already exists from
  earlier migration, resolved in previous commit)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Frontend: pass accessToken through McpAppIframe to authorize tool-call requests
- Frontend: include Authorization header in AppBridge oncalltool fetch
- Backend: register /v1/mcp/tool-call route
- Backend: rewrite v1_mcp_tool_call.go to query profile_mcp_installs instead of mcp_configs, matching actual MCP server storage

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add bridge.onmessage handler in McpAppIframe to receive messages
from HTML Apps via MCP ext-apps sendMessage() API. Text content
blocks are extracted and forwarded to the current chat thread as
user input, triggering a new agent run.

Files changed:
- McpAppIframe.tsx: register bridge.onmessage, onSendMessage prop
- ResourceUIPreview.tsx: pass through onSendMessage
- MessageList.tsx: add onSendMessage prop
- ChatView.tsx: wire sendMessage to MessageList and ResourcePreviewPanel
- ResourcePreviewPanel.tsx: pass through onSendMessage to McpAppIframe

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Third-party providers wrapping the Anthropic API (e.g. DeepSeek) may
require reasoning_content to be echoed back in conversation history.
Previously QuirkEchoReasoningContent was only registered for the OpenAI
SDK path; errors from such providers on the Anthropic SDK path were
misidentified as SymptomEmptyTextOnThinking, triggering the wrong fix
(adding an empty text block instead of reasoning_content).

- Add SymptomReasoningContentPassback to anthropicSymptoms
- Add QuirkEchoReasoningContent to anthropicQuirks
- Guard SymptomEmptyTextOnThinking to exclude reasoning_content errors
- Add SDK override for the new quirk in anthropic_sdk.go
- Update tests to use findAnthropicQuirk helper instead of hardcoded indices

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The project_meta_context feature was never fully wired up — the prompt
injection function (ApplyProjectContext) was dead code. Remove the entire
data path: DB column references, event emission from MCP executor, event
handlers in both cloud and desktop paths, RunContext field, and input
loader plumbing. The DB column itself is left in place (nullable,
harmless); migration 00198 already has a down migration for rollback.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ChatView was reading cached sources, artifacts, widgets, etc. when
loading historical messages, but missing readMessageResources. This
caused MCP HTML Apps to disappear after page refresh or reopening a
conversation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace inline iframe rendering with a compact entry card (icon, name,
preview button) for MCP App resources in the message stream. Full
rendering remains in the right-side panel.

Also includes worker debug logging for MCP tool discovery and execution.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Remove redundant header bar from MCP App in ResourcePreviewPanel
  (tab title and close button already handled by right panel tab)
- Remove internal uri bar via hideHeader prop
- Remove rounded border in fullscreen via noBorder prop
- Add width/height 100% to McpAppIframe container to fill panel space

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Use serverId instead of filename/uri for right panel tab titles,
so users see the MCP server name (e.g. "od") rather than the
generated artifact filename.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts:
#	src/services/worker/internal/llm/quirks_test.go
Add ImageGenModelSettingControl to web routing settings page, using
platform settings (image_generative.model) for persistence. Also increase
image model test timeout from 15s to 60s to accommodate slower image
generation APIs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- McpAppResource: add toolName/toolInput fields
- extractResources: accept optional tool name and arguments
- buildMessageResourcesFromAgentEvents: match tool-call events with
  tool-result events by toolCallId to propagate tool info
- useThreadSseEffect: track tool-call args via toolCallInfoRef,
  pass them to extractResources on tool-result; clear on run reset;
  populate cancelledToolCallIdsRef on run termination
- stream context: add cancelledToolCallIdsRef for run cancellation
- McpAppIframe: sendToolInput after oninitialized, sendToolCancelled
  when toolCancelled prop is true, teardownResource on unmount
- McpAppIframe: full HostContext (theme, locale, platform, userAgent,
  timeZone, containerDimensions, toolInfo) and HostCapabilities
  (openLinks, serverResources, logging, sandbox, message)
- McpAppIframe: support pip display mode, onrequestteardown,
  onupdatemodelcontext, onloggingmessage, ondownloadfile handlers
- ResourceUIPreview/ResourcePreviewPanel/MessageList: pass toolName
  and toolInput through to McpAppIframe

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The protocol expects permission fields as empty objects {} or omitted,
not booleans. Remove the permissions declaration since no sandbox
permissions are granted.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ayMode naming

- Add required inputSchema field to toolInfo.tool for MCP ext-apps protocol compliance
- Rename displayMode 'card' to 'fullscreen' across McpAppIframe, ResourceUIPreview, and MessageList
- Extract displayMode from tool results in agentEventProcessing and add to McpAppResource type
- Respect resource displayMode in MessageList rendering instead of hardcoding card mode

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…P App download handler types

- build-sidecar.mjs: use cgoEnabled for activity-record instead of hardcoded '0',
  fixing undefined idleSeconds/ActiveWindow on macOS (tracker_darwin.go requires CGO)
- McpAppIframe.tsx: fix ondownloadfile to match McpUiDownloadFileRequest protocol
  (params has contents array, not url/filename), add proper type narrowing for
  EmbeddedResource union type

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant