Repository navigation
fix(connector): complete the 16 incomplete connector implementations - #5394
Merged
qqeasonchen merged 3 commits intoSep 11, 2026
Merged
Conversation
All follow the new-architecture SourceConnector/SinkConnector contract (init/resume/poll/commit + init/put/commit); external-system logic ported from the master-branch (openconnect) implementations. - canal source: canal TCP client (getWithoutAck -> CloudEvent; ack on commit) - canal sink: all-or-nothing JDBC batch (rollback + throw on failure) - chatgpt source: HTTP prompt endpoint + OpenAI REST completion - dingtalk/lark/slack/wechat/wecom sources: platform webhook receivers (HMAC/SHA1/token signature verification, challenge handshakes) - mcp source/sink: JSON-RPC 2.0 notification receive + forward - openfunction source/sink: function output receive + HTTP trigger invoke - pravega source/sink: EventStreamReader / EventStreamWriter - prometheus sink: Pushgateway text-format push - spring sink: injectable EventForwarder bridge (fail-fast until wired) Dependencies added: canal.client 1.1.7, pravega-client 0.11.0, jackson-databind 2.18.0 (JSON plugins), mysql-connector-j compileOnly.
Overview (model, running, catalog, matrix) + one page per plugin (classes, source/sink behavior, config key tables from the actual props.getProperty defaults, run examples). Replaces the master-era website 03-connect docs (9 plugins) with full 23-plugin coverage matching the new-architecture contract.
Contributor
Author
|
Update: folded the connector docs into this PR so the implementation fix and its operator-facing docs land together (single review/merge unit).
|
qqeasonchen
added a commit
to qqeasonchen/eventmesh-site
that referenced
this pull request
Sep 11, 2026
…rsion Sync from apache/eventmesh develop docs/feature/connectors (the docs folded into PR apache/eventmesh#5394). Website adaptations: - README.md renamed to connectors.md (versioned_docs convention) - sidebar_position frontmatter on every page; category at 6.5 - internal links rebased one level deeper (feature/connectors/) - documentation-map.md gains the Connectors bullet
qqeasonchen
added a commit
that referenced
this pull request
Sep 17, 2026
Bring the single source of truth up to date with what actually landed: - SSE / WS push: unified ACK-tracked redelivery + DLQ now shared with long-polling via ReliableDispatcher; real-broker e2e suite in #5389 - Connector Runtime: all 23 plugins fully implemented since #5394 (the "rest are templates" note is stale); unit tests still cover only file/kafka/pulsar/rocketmq - A2A: task reaper + Meta-backed agent cards landed in #5346, quota classification in #5373; Beta promotion gated on Testcontainers E2E (#5340) instead of "pending" items - add the memory storage backend row (default, zero-dependency dev/CI; #5400) and mention it in the features list, both languages
qqeasonchen
added a commit
that referenced
this pull request
Sep 17, 2026
…in, A2A to 10108, 10205 gRPC reserved 8080/8081 collide with the most common dev-service ports (Tomcat, Jenkins, Spring Boot apps, proxies). Move back onto the project's own 1.x default port block, which keeps long-time users on familiar numbers:/n - traffic HTTP 8080 -> 10105 (all app protocols multiplex on it, incl. the legacy /eventmesh/* bridge) - admin HTTP 8081 -> 10106 - WebSocket 8082 -> 10107 (opt-in, unchanged default -1) - A2A Gateway 10105 -> 10108 (frees 10105 for the CORE runtime; the experimental gateway takes the adjacent 10108) - 10205 documented as RESERVED for a future gRPC protocol port (the 1.x default): the uni runtime does not ship a gRPC server yet, and the port must not be casually taken so a later gRPC addition lands on the historical number Touched: EventMeshApplication defaults, bin/start.sh, docker/Dockerfile(+.connector), K8s manifests + README, SDK CloudEventsClient default URL + javadocs, A2AClient / ExampleConstants / A2AGatewayDemo, e2e test literals, docs (quickstart, configuration, deployment, a2a) and eventmesh.properties comments. Configuration keys are unchanged - only defaults move; existing deployments pinning the old ports keep working. Also point image-smoke-test.yml at the new ports throughout (host = container, 10105/10106 - no offset mapping left). README / README.zh-CN quick-start port mentions updated too. Feature docs swept too (client-java, pubsub, streaming, introduction, index, admin-api, http-api, observability, connectors overview) - connector-local webhook ports are untouched. Fix a port-sweep typo (10100 -> 10105) in the documentation map. README / README.zh-CN capability-status refreshed against the current tree:/n- SSE/WS push: ACK-tracked redelivery + DLQ shared with long-polling (ReliableDispatcher); e2e-real-broker suite in #5389 - Connector Runtime: all 23 plugins fully implemented since #5394 (no template stubs left); 4 of 23 still carry the only unit tests - A2A Gateway: reaper + Meta-backed agent cards landed #5346, quota #5373; Beta promotion gated on Testcontainers E2E (#5340) - new Memory storage row (Beta, zero-dependency dev/CI default, #5400)
qqeasonchen
added a commit
to qqeasonchen/eventmesh
that referenced
this pull request
Sep 21, 2026
…bility table (en/zh) - Connector Runtime: 23 plugins complete since apache#5394, hardening in apache#5328 - A2A: the Testcontainers E2E promotion gate (apache#5340) closed 2026-09 - Recommendation wording Evaluate -> Usable; stale gate wording updated
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes?
Completes the 16 incomplete connector implementations under
eventmesh-connector-plugin/on the new architecture. All follow the new-architectureSourceConnector/SinkConnectorcontract (init/resume/poll/commit+init/put/commit); external-system logic is ported from the master-branch (openconnect) implementations where available.getWithoutAck→ CloudEvent; batch ack oncommit)EventStreamReadervia reader-group (native offsets; reconnect on rebalance)EventStreamWriter(auto scope/stream creation; write + join)EventForwarderbridge (fail-fast until the Spring context wires it)Design notes
HttpServerreceives platform callbacks into a buffer;poll()drains it — matching the ConnectorRuntime §8 poll/commit loop without threads inside the connector.HttpURLConnection; only canal (canal.client/protocol 1.1.7) and pravega (pravega-client 0.11.0) get real clients (both versions match master). JSON plugins addjackson-databind 2.18.0(already the repo-managed version).Verification
compileJavagreen on all 12 touched modules (JDK 21)checkstyleMain0 violations on all 12 modules (canonical ASF header, ImportOrder groups, VariableDeclarationUsageDistance)src/test/resourcesconfigs, no test classes)Follow-up (out of scope): unit tests per new implementation,
resume()offset wiring for push-style sources.Fixes #5395