Skip to content

fix(sandbox): guard the command path-translation regex with a segment boundary - #4110

Merged
WillemJiang merged 1 commit into
bytedance:mainfrom
AnnaSuSu:fix/command-path-segment-boundary
Jul 12, 2026
Merged

WillemJiang merged 1 commit into
bytedance:mainfrom
AnnaSuSu:fix/command-path-segment-boundary

Conversation

@AnnaSuSu

Copy link
Copy Markdown
Collaborator

Why

This is the last unguarded member of the family #4035 and #4053 fixed, mirrored into the other direction. I found it while enumerating that family for #4108 (the shared-helper extraction), and it is a behavior change, so it is here rather than folded into that refactor.

replace_virtual_paths_in_command matches the virtual root with no segment-boundary lookahead:

pattern = re.compile(rf"{re.escape(VIRTUAL_PATH_PREFIX)}(/[^\s\"';&|<>()]*)?")

The trailing group needs a / to consume anything. When the character after /mnt/user-data is -, ., _, a digit or a letter, the group matches empty and the bare root still matches. replace_virtual_path then takes its path == virtual_base branch — _thread_virtual_to_actual_mappings maps the virtual root to the three dirs' common parent — and substitutes the host directory, with the rest of the sibling name riding along:

cat /mnt/user-data-backup/secret.txt   ->   cat <host>/threads/t1/user-data-backup/secret.txt
ls  /mnt/user-data2                    ->   ls  <host>/threads/t1/user-data2

So a command naming a prefix sibling of the mount root is silently pointed at a real host directory outside the mount contract, and the agent reads or writes it. End-to-end on main, with a host sibling directory next to the thread's user-data:

$ cat /mnt/user-data-backup/secret.txt
  translated: cat <tmp>/user-data-backup/secret.txt
  bash says : HOST-ONLY SECRET

On this branch the path is left untranslated and the command fails honestly:

  translated: cat /mnt/user-data-backup/secret.txt
  bash says : cat: /mnt/user-data-backup/secret.txt: No such file or directory

bash runs every command through this function on the local sandbox, so this is not a narrow path.

What changed

One lookahead, mirroring the guard its siblings already carry:

pattern = re.compile(rf"{re.escape(VIRTUAL_PATH_PREFIX)}(?=/|$|[^\w./-])(/[^\s\"';&|<>()]*)?")

The class mirrors LocalSandbox._content_pattern's (?=/|$|[^\w./-]), not _command_pattern's shell-oriented (?=/|$|[\s\"';&|<>()]). Despite this running over a command string, a virtual root can legitimately be followed by : (PATH-style concatenation: PYTHONPATH=/mnt/user-data:/opt) or ,, both of which the shell class rejects — narrowing to it would stop translating paths that translate today. $ covers a command ending exactly at the root (ls /mnt/user-data).

Surface area

  • Sandbox — backend/packages/harness/deerflow/sandbox/tools.py

No documentation change needed: backend/AGENTS.md describes the virtual-path contract but makes no claim about this matcher's boundary semantics.

Bug fix verification

The delta set, enumerated in both directions (this is a narrowing change, so the risk is silently losing a translation). Old and new patterns run over a candidate list that deliberately includes the shapes I bet would not be affected:

newly_missed (old matched, new does not):
  '/mnt/user-data-backup/secret.txt'   old=['/mnt/user-data']  new=[]
  '/mnt/user-data2/report.txt'         old=['/mnt/user-data']  new=[]
  '/mnt/user-data.bak'                 old=['/mnt/user-data']  new=[]
  '/mnt/user-data_old/x'               old=['/mnt/user-data']  new=[]
  '/mnt/user-dataX'                    old=['/mnt/user-data']  new=[]

newly_caught (new matches, old did not): none
unaffected: 15/20

newly_missed is exactly the prefix-sibling family — the bug — and nothing else. The 15 unaffected include the cases the narrowing could plausibly have broken: PYTHONPATH=/mnt/user-data:/opt, echo '/mnt/user-data, done', cd /mnt/user-data; ls, echo /mnt/user-data\file, a quoted path with spaces, and a trailing slash.

Per-clause red check — each clause of the new lookahead broken in turn:

mutation red
drop the lookahead entirely (i.e. main) the 4 ..._does_not_rewrite_prefix_siblings cases
swap in _command_pattern's shell class the 2 ..._still_translates_genuine_paths cases for : and ,
delete only $ the 1 ..._still_translates_genuine_paths case for a command ending at the root

Each clause has an anchor that fails without it, and no test in the new set stays green under every mutation.

The ..._still_translates_genuine_paths cases are green on main as well — they do not guard the bug, they guard the fix from being narrowed too far, which is what the second and third mutations demonstrate.

Validation

  • cd backend && make lint — ruff check: all checks passed; ruff format --check: 830 files already formatted.
  • cd backend && make test — 7128 passed, 26 skipped, 8 failed. The same 8 fail on a clean origin/main checkout (pre-existing web_fetch cases — test_browserless_client, test_crawl4ai_tools, test_fastcrw_tools; no outbound DNS on this machine). Failure sets are byte-identical; this branch adds none.
  • End-to-end through real bash, nothing stubbed — the main vs branch transcript is quoted under Why above: on main the sibling command prints the host file's contents; on this branch it is never translated and fails with No such file or directory.

Related

AI assistance

Tool(s) used: Claude Code

How you used it: Claude Code assisted with enumerating the boundary-guard family across the sandbox layer (which is how this site surfaced), reproducing the host read through real bash, computing the delta set, and writing the regression tests. The contributor reviewed every line and takes responsibility for it.

  • I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

… boundary

replace_virtual_paths_in_command matches the virtual root with no
segment-boundary lookahead:

    re.compile(rf"{re.escape(VIRTUAL_PATH_PREFIX)}(/[^\s\"';&|<>()]*)?")

The trailing group needs a "/" to consume anything, so when the character after
/mnt/user-data is "-", ".", "_", a digit or a letter, the group matches empty
and the bare root still matches. The substitution then rewrites it to the
thread's host user-data directory and the rest of the sibling name rides along,
so a command naming a prefix sibling of the mount root is pointed at a real host
directory outside the mount contract:

    cat /mnt/user-data-backup/secret.txt  ->  cat <host>/user-data-backup/secret.txt

which reads the host file. This is the same defect as bytedance#4035 (reverse patterns)
and bytedance#4053 (masking patterns), mirrored into the virtual->host direction; it is
the last unguarded member of that family.

The boundary class mirrors LocalSandbox._content_pattern's rather than
_command_pattern's: a virtual root can legitimately be followed by ":"
(PATH-style concatenation) or ",", which the shell-oriented class rejects, so
narrowing to it would stop translating paths that translate today. "$" covers a
command ending exactly at the root.
@github-actions github-actions Bot added area:sandbox Sandboxed execution and docker/ risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/S PR changes 20-100 lines labels Jul 12, 2026
@WillemJiang
WillemJiang merged commit c82fba4 into bytedance:main Jul 12, 2026
11 checks passed
jtaynl added a commit to jtaynl/deer-flow that referenced this pull request Jul 12, 2026
lllyfff added a commit to qin-chenghan/deer-flow that referenced this pull request Jul 13, 2026
Brings in 4fd521e..2bd0f56: sandbox fixes (bytedance#4051/bytedance#4053/bytedance#4058/bytedance#4078/bytedance#4079/bytedance#4110),
CVE-2026-33128/bytedance#4089/bytedance#4093/bytedance#4106, subagents (bytedance#4056/bytedance#4059/bytedance#4088), skillscan (bytedance#4087),
channels (bytedance#4055/bytedance#4069/bytedance#4100), stream_bridge (bytedance#4071), frontend (bytedance#4058/bytedance#4068/bytedance#4094/
bytedance#4114/bytedance#4126), event-store serialization (bytedance#4077), streaming (bytedance#4082/bytedance#4085), mcp (bytedance#4084),
guardrails (bytedance#4067), and the memory fixes bytedance#4073/bytedance#4074/bytedance#4075/bytedance#4076/bytedance#4119.

Memory conflict resolution (upstream is still monolithic agents/memory/, we are
vendored backends/deermem/deermem/core/ - git rename-detected host->vendored):
- core/prompt.py, core/updater.py: keep ours (vendored with all ports); upstream's
  host-side bytedance#4119/bytedance#4074/bytedance#4075/bytedance#4076 changes are the same fixes already ported.
- core/queue.py: auto-merged cleanly (both sides had identical bytedance#4073 change).
- test_memory_queue.py: replaced the stale pre-bytedance#4073 test with upstream's three
  bytedance#4073 tests, adapted from host API (MemoryUpdateQueue()) to vendored (_queue()).
- test_memory_search.py: kept our vendored tests + added a vendored
  test_null_confidence_does_not_crash_sort (bytedance#4076 coverage).

Post-merge: 171 memory tests pass (+2 module-skipped), 7164 collected, no
import/collection errors. All 8 upstream memory fixes since the 8be7411
divergence are adapted (bytedance#4044/bytedance#4028/bytedance#4097/bytedance#4119/bytedance#4073/bytedance#4074/bytedance#4075/bytedance#4076).

Co-Authored-By: Claude <noreply@anthropic.com>
@AnnaSuSu
AnnaSuSu deleted the fix/command-path-segment-boundary branch July 20, 2026 02:01
marvin9551 pushed a commit to marvin9551/deer-flow that referenced this pull request Aug 21, 2026
… boundary (bytedance#4110)

replace_virtual_paths_in_command matches the virtual root with no
segment-boundary lookahead:

    re.compile(rf"{re.escape(VIRTUAL_PATH_PREFIX)}(/[^\s\"';&|<>()]*)?")

The trailing group needs a "/" to consume anything, so when the character after
/mnt/user-data is "-", ".", "_", a digit or a letter, the group matches empty
and the bare root still matches. The substitution then rewrites it to the
thread's host user-data directory and the rest of the sibling name rides along,
so a command naming a prefix sibling of the mount root is pointed at a real host
directory outside the mount contract:

    cat /mnt/user-data-backup/secret.txt  ->  cat <host>/user-data-backup/secret.txt

which reads the host file. This is the same defect as bytedance#4035 (reverse patterns)
and bytedance#4053 (masking patterns), mirrored into the virtual->host direction; it is
the last unguarded member of that family.

The boundary class mirrors LocalSandbox._content_pattern's rather than
_command_pattern's: a virtual root can legitimately be followed by ":"
(PATH-style concatenation) or ",", which the shell-oriented class rejects, so
narrowing to it would stop translating paths that translate today. "$" covers a
command ending exactly at the root.
jihtsan pushed a commit to jihtsan/dnx-deer-flow that referenced this pull request Aug 29, 2026
… boundary (bytedance#4110)

replace_virtual_paths_in_command matches the virtual root with no
segment-boundary lookahead:

    re.compile(rf"{re.escape(VIRTUAL_PATH_PREFIX)}(/[^\s\"';&|<>()]*)?")

The trailing group needs a "/" to consume anything, so when the character after
/mnt/user-data is "-", ".", "_", a digit or a letter, the group matches empty
and the bare root still matches. The substitution then rewrites it to the
thread's host user-data directory and the rest of the sibling name rides along,
so a command naming a prefix sibling of the mount root is pointed at a real host
directory outside the mount contract:

    cat /mnt/user-data-backup/secret.txt  ->  cat <host>/user-data-backup/secret.txt

which reads the host file. This is the same defect as bytedance#4035 (reverse patterns)
and bytedance#4053 (masking patterns), mirrored into the virtual->host direction; it is
the last unguarded member of that family.

The boundary class mirrors LocalSandbox._content_pattern's rather than
_command_pattern's: a virtual root can legitimately be followed by ":"
(PATH-style concatenation) or ",", which the shell-oriented class rejects, so
narrowing to it would stop translating paths that translate today. "$" covers a
command ending exactly at the root.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:sandbox Sandboxed execution and docker/ risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/S PR changes 20-100 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants