feat(dropbox): grep/rg search push-down via files/search_v2 - #568
Merged
Merged
Conversation
Recursive grep/rg on a Dropbox mount previously downloaded every file. With the new content_search / contentSearch config knob (off by default: full-text search is plan-gated and its index lags recent writes), both commands now ask /2/files/search_v2 which files contain the pattern's literal and download only those candidates. Output stays exactly GNU/ripgrep because the local scan still decides every match: - Core searchFiles pages search_v2 + search/continue_v2, dedups across pages, and reports the 10,000-match ceiling; narrowPaths maps path_lower/path_display back to mount paths under root_path, sorts narrowed candidates into sorted-readdir walk order, and rebases raw_path onto the scope spelling so labels match a walk's. - narrow_scope gates the push-down: literal (or regex-required-literal) single patterns only, recursive scans only, directory operands only, and never for output modes that must see every file (grep -v/-c, rg -v/--type/--glob). Empty/failed/truncated searches fall back to the full walk; binary-extension candidates are dropped to mirror the walk's skip; rg prunes hidden candidates segment-wise and forces walk-style filename labels. - Both wrappers keep the factory's default_provision so cost estimates are unchanged. - fix(grep, python): grep -Rl with a file operand now stats first and scans the file instead of readdir-walking it (GNU + TS parity); narrowed candidates exercised this path. - Fakes gain search_v2 + search/continue_v2 (case-insensitive substring over names and content — a superset of real token matching, which is what narrowing needs) with cursor paging; battery adapters enable the knob, so all dropbox/dropbox-root grep/rg cases now exercise the push-down live: 988/988 per target on both hosts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…matting The local TS battery ran against a stale mirage-node dist (only core was rebuilt), so the node DropboxResource never forwarded contentSearch and search narrowing was silently inactive on the TS host; CI's fresh build activated it and exposed two latent TS bugs python had already fixed: - grepFilesOnly walked file operands under -r (readdir on a narrowed file candidate -> ENOENT warnings, empty output). It now stats first and takes the single-file scan for file operands (GNU + python grep_files_only parity); regression tests in both languages. - rg's plain-line path delegates to grepGeneric, whose single-file body honors -H over -h, so the wrapper's forced label defeated -I suppression. Both wrappers now skip forcing H when -I is set; regression tests in both languages. Also formats the new files pre-commit never saw locally (they were untracked when it ran; --all-files only covers git ls-files) and settles two formatter fights: the provision calls are hoisted onto a shared dropboxResolveGlob const so Prettier/ESLint agree, and the test_grep_helper import gets grep_helper via a module import so yapf/isort converge. Verified with fresh core+node dists: dropbox/dropbox-root 988/988 on both hosts (narrowing live), ram/disk 2170/0, core vitest green, pre-commit converges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bytecii
force-pushed
the
feat/dropbox-search-pushdown
branch
from
July 19, 2026 11:06
3a8a39e to
ccc43d0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Recursive
grep/rgon a Dropbox mount previously walked the tree and downloaded every file. This adds an opt-in search push-down: withcontent_search=True(TScontentSearch: true), both commands first ask/2/files/search_v2which files contain the pattern's literal, then download and scan only those candidates. Output stays exactly GNU/ripgrep because the local scan still decides every match — search only narrows the candidate set.Follows the GitHub backend's narrow-then-scan blueprint and the Slack backend's availability-gate pattern, with three deliberate hardenings the GitHub narrow doesn't have:
-v/-c(grep) and-v/--type/--glob(rg) force the full walk — a narrowed superset would hide files those output modes must see.-lshort-circuit: Dropbox search is case-insensitive, so narrowed candidates are always re-scanned locally.Why the knob is off by default
Full-text content search is plan-gated (Professional/Essentials/Business+); on other plans
search_v2silently matches file names only, which would make a narrowed scan miss content matches. Dropbox's search index also lags recent writes. Docs spell out both caveats.Details
search_files/searchFilespagessearch_v2+search/continue_v2, dedups across pages (API may repeat results), and flags the 10,000-match ceiling;narrow_paths/narrowPathsmapspath_lower/path_displayback to mount paths underroot_path(case-insensitive scope filter, display-cased output), sorts candidates into sorted-readdir walk order (component-wise, not lexicographic), and rebasesraw_pathonto the scope's as-typed spelling so labels match a walk. Failed/empty/truncated searches fall back to the full walk.narrow_scope/narrowScope): knob on + recursive + single-line literal (regexes narrow on an extracted required literal) + all scope operands stat as directories. Unlike GitHub there is no scope-size gate: one search call plus targeted downloads beats a full walk at every scope size.default_provision, so cost estimates are byte-identical to before (prov_*battery cases).grep -Rlwith a file operand now stats first and scans the file instead of readdir-walking it — GNU behavior and TS parity; narrowed candidates exercised this latent gap.integ/server/dropbox.ts+dropbox_server.pyimplementsearch_v2/search/continue_v2with cursor paging; matching is case-insensitive substring over names and content — a superset of real token matching, which is exactly the contract narrowing needs.Verification
dropbox+dropbox-root988/988 on both hosts (push-down active); ram/disk/s3 regression targets clean on both hosts (python 3271/0 incl. s3; TS 2170/0, s3 skipped locally).spec/regenerated — no drift (registered command names unchanged).docs/python/resource/dropbox.mdx+docs/typescript/dropbox.mdx, resource-matrix note; python example extended with a write roundtrip + search demo.🤖 Generated with Claude Code