Pod token mint 403s with unverified-owner-process for every caller: the error text names two causes, neither is the real one, and its remedy deletes the pod #12958

Minting a pod dashboard credential is refused 403 for every caller on this host, while the pod itself is healthy and serving. The caller-side error text offers two causes. The pod's own audit log shows the refusal is a third cause the message never mentions, and both printed remedies are ruled out by that same log. One of those remedies deletes the pod's data directory.

Measured evidence

The pod's security_events.jsonl holds exactly 3 token.local records, and all 3 are:

outcome=denied  resources=unverified-owner-process  source=local-bootstrap

Zero non-loopback records. Zero invalid-secret records.

The gateway's pod_up surfaced it as:

pod 'kc-wt-wake-judge' refused the token mint over its API socket (HTTP 403)

A sandboxed CLI caller on the same host is stopped earlier, by a different gate:

withholding a credential for pod 'kc-wt-wake-judge': could not prove ownership of :7989
from the gateway pid record and the service manager's current MainPID

Both printed remedies are wrong

src/kiro_crew/pod/runtime.py:2342 names two causes and only two.

Cause 1, as printed: if the pod's worktree predates unix-socket admission on /api/token/local, its gateway 403s this transport whatever the secret, so update the worktree and restart the pod.

Ruled out twice. The worktree carries the unix-socket admission code from #9051 (_unix_peer_is_self, src/kiro_crew/dashboard/handlers/core.py:3013, applied at the top of api_token_local), and the audit log has zero non-loopback denials. The transport was admitted, not refused.

Cause 2, as printed: otherwise the gateway rejected the pod's .local_secret as stale from an earlier run, and restarting the pod regenerates both ends.

Ruled out. Zero invalid-secret denials, so the secret was accepted. The pod was also restarted at 18:48, and the very next mint 53 seconds later was refused identically.

The refusal comes from the third gate in api_token_local, local_owner_bootstrap_allowed (core.py:3055-3069), which returns code: member_owner_token_refused. The caller's message never mentions that gate, and the caller discards the code the pod sent.

Mechanism

local_owner_bootstrap_allowed (src/kiro_crew/member_memory_auth.py:131 on main) requires:

isinstance(pid, int)
and platform_compat.get_process_start_id(pid)
and (_verified_host_process(pid) or _gateway_spawned_app_backend(pid))

On Linux _verified_host_process is process_namespaces_match(pid, os.getpid()) is True, which compares the caller's user and mount namespaces against the pod gateway's own (src/kiro_crew/platform_compat.py:2231). _gateway_spawned_app_backend is false here, because a pod-mint caller is not an app backend the pod spawned.

Transport admission passed, so check_peer_is_self returned MATCH. The caller is therefore the same UID principal, and _request_peer_pid did return a pid. What remains failing is get_process_start_id(pid) or the namespace comparison.

Candidate cause, UNVERIFIED: the caller runs in a different user or mount namespace than the pod's gateway process, so the comparison cannot return True. I could not confirm this directly. From the agent sandbox every /proc/<pid>/ns/{user,mnt} outside my own process is unreadable, and systemctl --user answers Failed to connect to bus: Permission denied. My own namespaces are non-initial (user 4026533470, mnt 4026533475).

Why it matters

The pod answers health 200 and serves its bundle, yet every authenticated route on it is unreachable, so the live verification pods exist for cannot run. This blocked a QA recording campaign for about six hours across two token handoff attempts.

The printed remedy makes it worse. kirocrew pod down <name> && kirocrew pod up <name> deletes the pod's data directory, and cannot fix a caller-provenance refusal. Two restarts were performed on the strength of that text, and the refusal survived both.

Ask

Two separable pieces.

1. Report the cause actually hit. The pod already distinguishes all three cases in its SEL record and returns code: member_owner_token_refused in the body. runtime.py:2342 throws that away and prints two guesses instead. Reading the code and printing the matching remedy costs nothing, and would have prevented both restarts and the near-miss on the data directory.

2. Decide whether a same-principal caller in a different namespace may mint. Keep the fail-closed direction. The question is only whether namespace divergence alone should disqualify a caller the kernel already confirms is the same principal, and what positive proof should replace it.

Not duplicates

• pod credential attestation fails when /proc/<MainPID> is unreadable under the systemd user unit, withholding access to a healthy pod #9188 (open): same fail-closed family, different gate. It never reaches the request, because port_owner cannot attest via /proc/<MainPID>. This one passes ownership proof, transport admission and the secret check, then fails owner provenance.
• Dev Fleet open button always fails on Linux: pod ownership prover runs inside the sandboxed app backend, where lsof cannot attribute sockets #8551, Browser view never starts where lsof cannot attribute sockets: the healthy child is reaped and misreported as a port squatter #11887 (closed): lsof socket attribution.
• Carry the app name into the local token mint audit and payload #10427 (open): enhancement threading an app name through the same audit record, not this refusal.
• Send X-Local-Secret over the main gateway's unix socket in cli_server.py (deferred from PR #9051) #11091 (open): deferred X-Local-Secret over the main gateway's socket.

Environment

Linux. Pod kc-wt-wake-judge on :7989, health 200. Pod worktree at 3ddc358ee3, main f71094d366. The gate and the message are identical on main, so this is not branch-scoped.