Skip to content

Latest commit

 

History

History
201 lines (170 loc) · 11 KB

File metadata and controls

201 lines (170 loc) · 11 KB

Ruby SDK conformance report

Where the Ruby SDK stands against keycard-sdk-spec. Generated by hand from the suites in this repo; every number below is reproducible with the commands in the last section.

Summary

Gems keycardai-oauth, keycardai-mcp, keycardai-a2a
Capability specs covered 19 of 22, 2 excluded on purpose, delegated-access/as-itself an open gap
Partial coverage authorization-code-pkce v3 rows 8 to 11 (the stateless web-app begin/complete pair) are unshipped; Ruby covers the flow's building blocks and the loopback authenticate
Conformance examples 173 (plus 6 load/sanity checks, 179 total)
Live-zone integration rows 13 of 13 passing against a real zone
A2A delegation checks 11 of 11 hermetic, plus a live run
MCP server end-to-end checks 8 of 8 hermetic, against the official mcp gem
Ruby versions 3.2, 3.4, 4.0 in CI
Lint RuboCop clean

Every conformance suite maps one-to-one to a spec's Testing section: each it is labelled with the row number it implements, so a reviewer can diff the suite against the spec table directly. Suites also carry a few extra examples beyond the spec rows (construction errors, cache eviction, credential pairing), which are labelled without a row number.

Coverage

keycardai-oauth

Spec Suite Examples
jwt-jwks/jwt-signing-and-verification jwt_signing_and_verification_spec.rb 18
jwt-jwks/jwks-caching jwks_caching_spec.rb 11
oauth-client/authorization-server-discovery authorization_server_discovery_spec.rb 9
oauth-client/userinfo userinfo_spec.rb 8
oauth-client/client-credentials client_credentials_spec.rb 6
oauth-client/token-exchange token_exchange_spec.rb 10
oauth-client/dynamic-client-registration dynamic_client_registration_spec.rb 5
oauth-client/authorization-code-pkce authorization_code_pkce_spec.rb + authenticate_flow_spec.rb 17
application-credentials/client-secret client_secret_spec.rb 6
application-credentials/web-identity web_identity_spec.rb 8
application-credentials/workload-identity workload_identity_spec.rb 10
delegated-access/access-context access_context_spec.rb 19
delegated-access/impersonation impersonation_spec.rb 5
multi-zone-and-ops/multi-zone-support multi_zone_support_spec.rb 6

keycardai-mcp

Spec Suite Examples
server-bearer-auth/bearer-token-verification-middleware bearer_token_verification_middleware_spec.rb 10
server-bearer-auth/oauth-metadata-endpoints oauth_metadata_endpoints_spec.rb 8
server-bearer-auth/route-level-auth-gating route_level_auth_gating_spec.rb 5
delegated-access/grant-decorator grant_decorator_spec.rb 5

keycardai-a2a

Spec Suite Examples
a2a/a2a-delegation a2a_delegation_spec.rb 7

Live-zone results

examples/mcp-server/bin/live-e2e executes the specs' Integration Tests rows against a real Keycard zone. Objects are provisioned by examples/mcp-server/bin/provision, which drives the Management API through keycard agent api using the operator's own signin session.

Spec Row Result
authorization-server-discovery 1: discover against a live zone pass
jwks-caching 1: resolve a live signing kid, second call cached pass
client-credentials 1: acquire a token for a registered client pass
client-credentials 2: an invalid client secret is invalid_client pass
token-exchange 2: a malformed subject token is rejected (invalid_request) pass
token-exchange 3: an unregistered resource is rejected (invalid_target) pass
impersonation 1: impersonate a user; sub is the target pass
token-exchange 1: a zone-issued token is exchangeable for a second resource pass
grant-decorator 1: the grant path exchanges an inbound token for every resource pass
jwt-signing-and-verification 2: verify a real zone token, second verify cached pass
bearer-token-verification-middleware 1: a real token through the Rack middleware pass
access-context 1: multi-resource grant populates an AccessContext pass
access-context 2: one failing resource yields partial_error pass

Two further end-to-end suites run without any zone: examples/mcp-server/bin/selftest (8 checks, an MCP server on the official mcp gem behind a stub zone with real RSA keys) and examples/a2a-delegation/bin/selftest (11 checks, two agents whose stub zone performs a genuine RFC 8693 exchange).

Findings against the spec

Raised by implementing the contract from scratch and running it live. None are Ruby-specific.

  1. impersonation.md expects an act chain that no live zone issues. Unit row 1 and integration row 1 both require the issued token to carry an act claim naming the impersonating service, which the contract says the authorization server derives from client authentication. In svc-sts (packages/a3-oauth2/src/exchange/token.js) actor information is populated only inside the if (actorToken) branch, and the impersonation flow deliberately sends no actor_token, so no act is ever emitted. A live token carries aud, client_id, exp, iat, iss, jti, sub and nothing else. Owned by another team. The Ruby suites assert sub and report act rather than asserting it; the hermetic stub zone does emit act, so the contract's intent stays covered.

  2. Re-exchange requires the calling client to own the aud resource, and nothing specifies it. svc-sts compares resource.application_id against the authenticated client before accepting a zone-issued access token as a subject_token, and rejects a mismatch with invalid_grant ("Client is not allowed to exchange token for this resource"). Its own comment calls this "a structural invariant (not a policy decision)". No spec mentions it, yet it decides whether onward delegation works at all: with the field unset every exchange fails while impersonation keeps working, so it presents as a token problem rather than a provisioning one. Verified by setting it and watching the same exchange start succeeding. Belongs in token-exchange.md.

  3. A malformed subject_token yields invalid_request, not the invalid_grant the spec lists for a rejected subject token. Pinned as an assertion in the live suite.

    An earlier reading of these two is retracted. The refusal in finding 2 was described as an impersonation-specific anti-laundering rule; it is neither. The check applies to every zone-issued access token, the substitute-user token type is the unsigned input to impersonation rather than anything a zone issues, and with ownership set an impersonated token re-exchanges normally. Finding 3 was also paired with a claim that a zone never emits invalid_target; it does, for an unregistered target resource, exactly as the spec says.

Two spec-internal inconsistencies also surfaced while implementing jwt-signing-and-verification.md, both resolved by following the Divergences section over the body: the types table gives clock_skew a 60s default while Divergences records the canonical behaviour as exact comparison with no leeway, and the Contract lists only exp and client_id as required claims while Divergences records the full RFC 9068 set that TS and Python converged on. Ruby implements zero skew and the full claim set.

Three further items surfaced while implementing the OIDC discovery fields, UserInfo, and the spec-version 3 authorization-code revision:

  1. authorization-code-pkce.md keeps an optional resource in its code-exchange inputs table while spec-version 3 removes resource input from completion. The two read as a contradiction for a gem whose only code exchange is the building block the flow composes. Ruby resolves it by dropping the parameter from the exchange entirely, so no caller can send a value the authorization server ignores. Python (operations/_authorize.py) and TypeScript (src/pkce.ts) still accept resource on their standalone exchange, so this is a live divergence, not a settled contract.
  2. Both Divergences tables are stale in the direction of "nobody ships this". authorization-server-discovery.md records that no SDK types userinfo_endpoint / end_session_endpoint, and userinfo.md records that no SDK implements the capability, yet Python shipped both in keycardai-oauth 0.22.0 and TypeScript in @keycardai/oauth 0.21.0. userinfo.md also carries no ruby entry in its packages front matter.
  3. delegated-access/as-itself.md has no Ruby coverage and is not an exclusion. It postdates this report's first pass and was missing from the counts entirely; it is now carried above as an open gap.

Deliberate exclusions

Spec Why
application-credentials/eks-workload-identity Folded into workload-identity as a deprecated alias in the sibling SDKs. Ruby is new and ships no deprecated surface; FileTokenSource covers the contract, including the same four-variable discovery list at exact parity.
multi-zone-and-ops/per-user-token-cache Bound to @keycardai/cloudflare; no Ruby counterpart.
base.md Cross-cutting terminology and principles, no Testing table. Its rules are honoured throughout, notably constructor-injected configuration with no implicit environment reads.
framework-integrations/* Governance documents rather than generatable contracts. The Rack integration is judged against the FRAMEWORK-INTEGRATIONS.md checklist instead.

Every capability is now proven live, including the production grant path. It exchanges an inbound caller token for one token per downstream resource, and an impersonated token stands in for the verified inbound token, which is what makes the row headless: no browser login is involved. Reaching it needed the zone's resources to carry an application_id, since re-exchange is gated on the calling client owning the resource in the token's aud.

Idiom decisions

Ruby-specific expression choices, with the reasoning, are in idiom-profile.md. The load-bearing ones: sync-only with thread-safe caches, keyword arguments, Keycardai::Error as the taxonomy root, Rack as the framework seam, no implicit environment reads except the blessed FileTokenSource discovery list, and get_ prefixes dropped in favour of predicates ending in ? while set_* mutator names are kept because they are contract.

Reproducing

bundle install
bundle exec rake                      # 179 examples + RuboCop

cd examples/mcp-server
bin/selftest                          # 8 checks, no zone needed
ZONE=<zone-id> ORG=<org-id> bin/provision
bin/live-e2e                          # the 12 live rows

cd ../a2a-delegation
bin/selftest                          # 11 checks, no zone needed
ZONE=<zone-id> ORG=<org-id> bin/live