fix(cron): propagate cleanupCliLiveSessionOnRunEnd to isolated cron CLI branch - #97227

What Problem This Solves

Isolated cron CLI workers (src/cron/isolated-agent/run-executor.ts at line ~306) call runCliAgent without passing cleanupCliLiveSessionOnRunEnd, so worker processes spawned by CLI live sessions during cron runs are never cleaned up after the run completes. Over time, stale worker processes accumulate on the host, causing high load and slow responses.

The cleanup contract already exists in the CLI runner types and implementation (cli-runner/types.ts, cli-runner.ts), but was not wired for the isolated cron CLI code path.

Fixes #76171

Why This Change Was Made

Added cleanupCliLiveSessionOnRunEnd: params.job.sessionTarget === "isolated" to the runCliAgent call in src/cron/isolated-agent/run-executor.ts. This uses the existing session-target check ("isolated" when the job runs in isolated mode) to signal that the CLI live session should be cleaned up at run end — exactly the same pattern used elsewhere in the codebase.

No other changes are needed: the CLI runner already handles the cleanup when this flag is set.

User Impact

Users running cron jobs with isolated session targets (sessionTarget: "isolated") would previously see growing process counts and degrading host performance after repeated cron cycles. After this fix, each isolated cron CLI worker cleans up its live sessions at run end, preventing stale process accumulation.

No impact on non-isolated cron jobs or other run modes.

Fix scope and issue relationship

This PR fully fixes #76171 for the reported symptom (stale worker process accumulation). The canonical root cause — confirmed by ClawSweeper source-level review (June 26, 2026), contributor YusukeIt0, and this PR's own analysis — is that cleanupCliLiveSessionOnRunEnd was not propagated to the isolated cron CLI branch in runCliAgent. The flag triggers:

cleanupCliLiveSessionOnRunEnd: true
  → cli-runner.ts: closeClaudeLiveSessionForContext(context)
    → claude-live-session.ts: closeLiveSession(session, "restart")
      → session.managedRun.cancel("manual-cancel")
        → supervisor.ts: adapter.kill("SIGTERM") → (5s) → adapter.kill("SIGKILL")
          → Worker subprocess TERMINATED ✓

The embedded cron branch already had its corresponding cleanup (cleanupBundleMcpOnRunEnd → retireSessionMcpRuntimeForSessionKey, which is session-scoped and safe for concurrent runs). The only gap was the CLI branch missing its live-session cleanup flag.

Explicit non-goal: cleanupBundleMcpOnRunEnd was intentionally NOT added to the CLI branch. In the CLI runner (cli-runner.ts:476-482), this flag calls closeMcpLoopbackServer() — a process-global operation that would close the shared loopback HTTP server, breaking concurrent cron runs' MCP tool calls. The embedded runner's cleanupBundleMcpOnRunEnd uses a different, session-scoped mechanism (retireSessionMcpRuntimeForSessionKey). Adding process-level MCP cleanup to the CLI branch would introduce a worse bug than the one being fixed. Safe MCP runtime cleanup for CLI cron paths requires a session-scoped approach (tracked by PR #85241) and is not necessary to resolve the worker accumulation reported in #76171 — the stale workers are Claude CLI subprocesses, not MCP server processes.

Evidence

1. Runtime proof: isolated cron CLI run through the changed run-executor.ts path

Methodology: Temporary process.stderr.write markers were inserted at three points — (a) src/cron/isolated-agent/run-executor.ts:306 (CLI branch entry), (b) src/agents/cli-runner.ts:469 (cleanup branch entry), and (c) src/agents/cli-runner.ts:476 (cleanup complete). A claude-cli CLI backend was configured pointing at the local Claude Code installation (/usr/local/node24131/bin/claude, v2.1.187). An isolated cron job (sessionTarget=isolated, model=claude-cli/claude-sonnet-4-6) was created and executed via the gateway. Markers were removed from final commits.

Build and verify markers in dist:

$ pnpm build
$ grep -c 'E2E-76171' dist/cli-runner-*.js dist/*executor*.js
dist/cli-runner-6hFSl0IH.js:2          ← cleanup markers in binary
dist/run-executor.runtime-0I2wflpu.js:1 ← CLI branch marker in binary

Create and run isolated cron job:

$ node dist/index.js cron add --token <redacted> \
    --name "e2e-proof-76171-cron" --agent main_assistant \
    --model claude-cli/claude-sonnet-4-6 --session isolated \
    --message "say hello in one word" --timeout-seconds 30 \
    --delete-after-run --cron "0 0 1 1 *"
{
  "id": "337a5bb6-...",
  "sessionTarget": "isolated",
  "payload": { "kind": "agentTurn", "model": "claude-cli/claude-sonnet-4-6" }
}

$ node dist/index.js cron run 337a5bb6-... --token <redacted>
{ "ok": true, "enqueued": true }
# The cron job executes asynchronously inside the gateway process.
# Output appears in the gateway's stderr.

Gateway stderr — the changed run-executor.ts CLI branch entered, cleanup triggered, cleanup completed:

[E2E-76171] CRON EXECUTOR CLI branch: jobId=337a5bb6-... sessionTarget=isolated provider=claude-cli
[E2E-76171] CRON CLI cleanup TRIGGERED: sessionId=08268d8d-... trigger=cron provider=claude-cli
[E2E-76171] CRON CLI cleanup COMPLETED: sessionId=08268d8d-...

OpenClaw production log — Claude live session lifecycle confirms worker spawned, ran, and exited:

claude live session start: provider=claude-cli model=claude-sonnet-4-6 activeSessions=1
claude live session turn:  provider=claude-cli model=claude-sonnet-4-6 durationMs=5036 ...
claude live session close: provider=claude-cli model=claude-sonnet-4-6 reason=restart

Complete code path verified at runtime (all three markers confirmed + production log):

run-executor.ts:302  → isCliProvider(claude-cli) = true → CRON EXECUTOR CLI branch ENTERED  ← 标记1
run-executor.ts:313  → cleanupCliLiveSessionOnRunEnd: params.job.sessionTarget === "isolated"
                    → "isolated" === "isolated" → true
cli-runner.ts:467    → if (cleanupCliLiveSessionOnRunEnd === true) → ENTERED              ← 标记2
                    → closeClaudeLiveSessionForContext(context)
                       → claude-live-session.ts: closeLiveSession(session, "restart")
                          → session.managedRun.cancel("manual-cancel")
                             → supervisor.ts: adapter.kill("SIGTERM") → worker exits
                       → "claude live session close: reason=restart"                       ← 生产日志
cli-runner.ts        → closeClaudeLiveSessionForContext returned → COMPLETED              ← 标记3

This is the exact code path changed by this PR: an isolated cron agent turn with a CLI provider flows through run-executor.ts (where the fix lives) → runCliAgent (which now receives cleanupCliLiveSessionOnRunEnd: true) → the cleanup branch → the worker subprocess is signaled and exits cleanly. Before the fix, the "CRON EXECUTOR CLI branch" marker would still fire, but "CRON CLI cleanup TRIGGERED" would NOT — closeClaudeLiveSessionForContext was never called, leaving the worker alive.

Temporary config used for proof (added to ~/.openclaw/openclaw.json; claude-cli CLI backend is a bundled Anthropic plugin):

// models.providers — added claude-cli model catalog entry
"claude-cli": {
  "baseUrl": "cli://claude", "api": "openai-responses",
  "models": [{ "id": "claude-sonnet-4-6" }]
}
// agents.defaults.cliBackends — pointed at local Claude Code binary
"claude-cli": { "command": "/usr/local/node24131/bin/claude" }
// agents.defaults.models — alias for model selection
"claude-cli/claude-sonnet-4-6": { "alias": "sonnet" }

(Config retained; useful for future CLI-backend testing.)

2. Inner boundary safety proof (P1 addressed)

Question: Does running cleanup inside runCliAgent break CLI interim retries, where a second runCliAgent call reuses the same session?

Answer: No. Each runCliAgent call creates a fresh context via prepareCliRunContext at line 454—455, which in turn creates a fresh ClaudeLiveSession spawn. The cleanup key is buildClaudeLiveKey(context) keyed on the session identity; after the first call's cleanup, the key is removed from liveSessions, so the second call's spawn is completely independent.

runCliAgent (1st call, interim "on it")
  → prepareCliRunContext(params) → fresh context_1
  → runPreparedCliAgent(context_1) → spawn Claude CLI subprocess A
  → closeClaudeLiveSessionForContext(context_1) → kills subprocess A
  → liveSessions.delete(context_1_key)

[interim ack detected → retry requested]

runCliAgent (2nd call, continuation prompt)
  → prepareCliRunContext(params) → fresh context_2 (NEW, independent)
  → runPreparedCliAgent(context_2) → spawn Claude CLI subprocess B
  → closeClaudeLiveSessionForContext(context_2) → kills subprocess B

3. CLI interim retry test coverage (P1 addressed)

Two new tests added to src/cron/isolated-agent/run.interim-retry.test.ts:

Test A — "passes cleanupCliLiveSessionOnRunEnd on both the initial and retry CLI runs":

1. Sets isCliProviderMock = true (CLI path)
2. First runCliAgent returns "On it, grabbing…" (interim ack)
3. Second runCliAgent returns concrete result
4. Asserts cleanupCliLiveSessionOnRunEnd === true on both calls

Test B — "still passes cleanupCliLiveSessionOnRunEnd when the first turn is already a concrete result (no retry)":

1. Sets isCliProviderMock = true (CLI path)
2. runCliAgent returns concrete result (no retry needed)
3. Asserts cleanupCliLiveSessionOnRunEnd === true on the single call

$ pnpm test src/cron/isolated-agent/run.interim-retry.test.ts

 ✓ passes cleanupCliLiveSessionOnRunEnd on both the initial and retry CLI runs
 ✓ still passes cleanupCliLiveSessionOnRunEnd when the first turn is already a
   concrete result (no retry)

 Test Files  1 passed (1)
      Tests  7 passed (5 existing + 2 new)

4. Code-level proof: before (main) vs after (this fix)

src/cron/isolated-agent/run-executor.ts — CLI branch:

BEFORE (main):  const result = await runCliAgent({
                  sessionId: ...,
                  sessionKey: ...,
                  ...  // NO cleanup flag — workers never exit
                });

AFTER (fix):    const result = await runCliAgent({
                  sessionId: ...,
                  sessionKey: ...,
                  cleanupCliLiveSessionOnRunEnd: params.job.sessionTarget === "isolated",
                  ...  // cleanup flag present — workers exit cleanly
                });

5. Cleanup contract (already exists, now wired up)

src/agents/cli-runner/types.ts:144: cleanupCliLiveSessionOnRunEnd?: boolean

src/agents/cli-runner.ts:467-474:

if (params.cleanupCliLiveSessionOnRunEnd === true) {
  const { closeClaudeLiveSessionForContext } =
    await import("./cli-runner/claude-live-session.js");
  await closeClaudeLiveSessionForContext(context);
  // → closeLiveSession → session.managedRun.cancel("manual-cancel")
  // → supervisor.ts: adapter.kill("SIGTERM") → worker exits
}

src/agents/cli-runner/claude-live-session.ts:441-464:

function closeLiveSession(session, reason, error?) {
  session.closing = true;
  liveSessions.delete(session.key);
  session.managedRun.cancel("manual-cancel");  // ← kills child process
  void cleanupLiveSession(session);
}

src/process/supervisor/supervisor.ts:232-239:

cancelAdapter = (_reason) => {
  adapter.kill("SIGTERM");
  setTimeout(() => {
    adapter.kill("SIGKILL");  // force kill after 5s grace
  }, 5000).unref();
};

6. Process cleanup mechanism proof

A real OS process was spawned to validate the kill chain:

$ bash .workflow-custom/scripts/proof-76171-process-cleanup.sh

--- Step 1: Spawning dummy worker process (simulates isolated cron CLI worker) ---
Worker PID: 3163834
Status: ALIVE (verified via kill -0)

--- Step 2: Simulating managedRun.cancel("manual-cancel") ---
Sending SIGTERM...
Process exited cleanly via SIGTERM

--- Step 3: Process confirmed DEAD ---

✓ Cleanup mechanism verified:
  1. cancel("manual-cancel") called
  2. SIGTERM sent to process group
  3. Process terminated

7. Test evidence: CLI branch cleanup flag assertion

$ pnpm test src/cron/isolated-agent/run.session-key-isolation.test.ts

 ✓ uses a run-scoped key for CLI isolated cron execution
   → asserts runRequest.cleanupCliLiveSessionOnRunEnd === true  ← NEW assertion
 ... 5 passed (5)

8. Test evidence: closeClaudeLiveSessionForContext invoked when flag is set

$ pnpm test src/agents/cli-runner.before-agent-reply-cron.test.ts

 ✓ can close temporary CLI live sessions after a run
   → passes cleanupCliLiveSessionOnRunEnd: true to runCliAgent
   → asserts closeClaudeLiveSessionForContextMock called exactly once
 ... 13 passed (13)

9. Full test suite (all paths)

$ pnpm test src/cron/isolated-agent/run.session-key-isolation.test.ts \
         src/cron/isolated-agent/run.interim-retry.test.ts \
         src/agents/cli-runner.before-agent-reply-cron.test.ts

 Test Files  3 passed (3)
      Tests  25 passed (25)

10. Type check and lint

• pnpm check:test-types — passed
• No lock file changes

Behavior addressed: Isolated cron CLI workers now set cleanupCliLiveSessionOnRunEnd when sessionTarget === "isolated", ensuring Claude CLI subprocesses (the zombies reported in #76171) are terminated after each run via the ProcessSupervisor's SIGTERM→SIGKILL mechanism. Proof verified at three independent levels: live gateway cron execution, built binary inspection, and full test suite (25 tests across 3 files).