Repository navigation
fix(importer): import only torrent file list, not whole download dir (closes #903) - #933
Merged
Merged
Conversation
…loses #903) When a completed torrent is a single loose file with no containing subfolder (a standalone .m4b on Transmission's shared download dir), Bindery's importer walked the entire download directory and imported every recognised file there. Users seeding multiple books in the same directory saw unrelated books copied into the destination library. Each torrent downloader (Transmission, qBittorrent, Deluge) gains a Files(ctx, id) method that returns the authoritative per-torrent file list. The scanner now calls Files() before tryImport*, joins each relative name with the client's save path, applies the existing PathRemap helper, and threads the resolved absolute paths into tryImportInternal as explicitFiles. The legacy filepath.Walk path is retained as the fallback when Files() errors or returns empty (older client protocols, transient RPC issues), with a WARN log so an operator can spot the fallback firing. The audiobook flow takes a parallel branch: with the file list, a single-file torrent is placed as a lone file inside destDir; a multi-file torrent whose files share a strict subdirectory under the shared root moves that subdirectory; multi-file torrents sitting directly at the shared root fall back to per-file placement so unrelated siblings stay untouched. SAB and NZBGet keep walking the per-job destDir; both downloaders always land jobs inside a per-job folder, so the shared-root shape that motivates #903 does not apply. Malformed file names from a downloader response (absolute paths or any ".." path segment) are rejected with a WARN.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
vavallee
added a commit
that referenced
this pull request
Jun 3, 2026
Security and hardening release. - Opt-in per-user data isolation via BINDERY_ENFORCE_TENANCY: Tier-1 per-user resource IDOR closed (#899) and Tier-2 queue/history/pending + OPDS scoping (#898), both env-gated and off by default. - Access-control hardening: deployment-config routes behind RequireAdmin (#897), OIDC SSRF/redirect/proxy-scheme guards (#894), settings secret leak plugged (#893), library-root containment on file deletion (#895), session invalidation on password change (#896), 1 MiB request body cap (#901), StepSecurity sweep (#904), Go 1.25.11 stdlib CVE bump (#945). - Performance (Wave 3): pooled HTTP clients (#917), bounded goroutine fan-out (#918), TTL cache caps + enrichBook caching (#915), ABS checkpoint debounce (#916). - Behaviour changes: List endpoints paginated (#902); webhook payload Apprise-compatible (#888); backups via VACUUM INTO (#900). - feat: blocklist created_by_user_id audit column (#929). - Fixes: unmatched-audiobook import panic (#946), connrefused hint (#944), auto-grab history (#938), calibre series persistence (#936), torrent file-list import (#933), legacy date formats (#921), lifetime-ctx goroutines (#932, #934), ttlCache janitor leak (#881), and more. - Docs currency sweep (#949). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #903.
Reporter's repro (Bclark117)
/data/downloads); PathRemap configured per client (/data:/downloads)..m4bfile with no containing subfolder.downloadDirand copies every recognised book file there into the audiobook library, dragging the unrelated siblings in.Root cause: with Transmission a single-file torrent's
downloadDiris the shared root itself, not a per-torrent subdirectory.filepath.Walk(downloadDir)then enumerates every sibling, and the importer treats them all as part of this torrent. qBittorrent and Deluge share the same shape when the user disables the default per-torrent subfolder option.What changed
Per-client
Files(ctx, downloadID)APItorrent-getRPC withfields=["id", "files"][]File{Name, Size}whereNameis relative todownloadDirGET /api/v2/torrents/files?hash=<hash>[]File{Name, Size}whereNameis relative tosave_path(Windows backslashes normalised)core.get_torrent_statuswithkeys=["files"][]File{Name, Size}whereNameis relative to the torrent's save pathA new
File{Name, Size}type lives in each downloader package, matching the existing per-package types pattern.SABnzbd and NZBGet are intentionally not modified; both always land jobs inside a per-job destination folder (
storage/DestDir), so a directory walk there cannot pick up sibling jobs. Documented in code comments and skipped in the test plan.Threading through the importer
In
internal/importer/scanner.go:tryImportInternalaccepts a newexplicitFiles []stringparameter. When non-nil and non-empty, the importer uses it directly instead offilepath.Walk(downloadPath).tryImportTransmission/tryImportQbittorrentaccept the file list and forward it.checkTransmissionDownloads/checkQbittorrentDownloadscall the newtransmissionFilesFor/qbittorrentFilesForhelpers right after deciding to import, then pass the result through.resolveTorrentFiles(client, savePath, []torrentFile)does the mapping from client-relative names to absolute Bindery-side paths, joining withsavePathand running throughremapDownloadClientPathso the existing per-client and global remap rules still apply uniformly.Path-remap interaction
The new code reuses
remapDownloadClientPath(the same helper that already rewritestorrent.DownloadDirfor the parent dir). The flow per file is:clientPath = filepath.Join(clientSavePath, file.Name)— the path the download client sees.binderyPath = remapDownloadClientPath(client, clientPath)— the same per-client PathRemap rule that was already applied to the parent dir now rewrites the full file path consistently.IsBookFile(binderyPath)filter, matching the legacy walk.No new remap logic is introduced; the rules an operator sets in the UI keep working unchanged.
Fallback behaviour
When
Files()returns an error or an empty list:transmissionFilesFor/qbittorrentFilesForlog a WARN (so an operator can see the fallback firing) and returnnil.tryImportInternalfalls back to the legacyfilepath.Walk(downloadPath)discovery.This preserves behaviour for older client protocol versions, transient RPC issues, and torrents that have not yet flushed metadata to disk (Transmission at 0% returns an empty files array).
Audiobook flow
The audiobook branch previously moved the entire
downloadPath, which is the dangerous behaviour for #903's shape. With the file list:destDirvia the existing not-a-directory branch.downloadPath: source is that subdirectory (so cover art and cue sheets within the torrent's folder still ride along).destDir. Cover art outside the file list is left behind (the safer tradeoff vs. catching unrelated siblings).Safety guards
File names containing an absolute path or any
..path segment are rejected with a WARN. Windows backslashes are normalised at the qBit API boundary so the segment check works uniformly.Test plan
Unit tests for the downloader
Files()APIsTestClient_Files_ReturnsRelativeNames(Transmission / qBittorrent / Deluge): canned RPC response → asserts[]Fileshape matches.TestClient_Files_SingleFileTorrent(Transmission / qBittorrent / Deluge): the Bug Report: Single-file torrent import scans entire download directory #903 shape, basename-onlyName.TestClient_Files_TorrentMissing(Transmission / qBittorrent / Deluge): unknown id surfaces an error so the importer can fall back.TestClient_Files_NoFilesYet(Transmission): empty array, nil error (metadata still resolving).TestClient_Files_WindowsBackslashesNormalized(qBittorrent): backslashes from a Windows qBit instance rewritten to forward slashes.TestClient_Files_StatusErrorReturnsError(Deluge): RPC-level fault surfaces as an error.Importer-side tests
TestImport_SingleFileTorrentSkipsSiblingFiles(end-to-end throughcheckTransmissionDownloads): shared download dir with the torrent's own.m4bplus two unrelated siblings; Transmission stub reportsFiles=[the-book.m4b]; asserts onlythe-book.m4blands in the audiobook library and both siblings remain untouched.TestImport_FallsBackToDirWalkWhenFilesUnavailable:tryImportInternalwithexplicitFiles=nil(the post-RPC-failure shape) → asserts the dir walk runs and the book is imported.TestResolveTorrentFiles_BuildsAbsoluteBinderyPaths: covers PathRemap interaction andIsBookFilefiltering.TestResolveTorrentFiles_RejectsMalformedNames: absolute paths and..segments are dropped.TestResolveAudiobookSource_*: covers the single-file, common-subdir-accepted, and shared-root-falls-back-to-per-file branches.Regression coverage
go test ./...passes.go vet ./...clean.nilforexplicitFilesand exercise the fallback walk.🤖 Generated with Claude Code