Skip to content

rhom6us/FastMove

Repository files navigation

FastMove

High-performance Windows folder mover that outperforms XCOPY and ROBOCOPY.

FastMove uses kernel-level CopyFileEx P/Invoke with parallel I/O, unbuffered large-file transfers, and a dedicated USB/removable-drive pipeline to move directories as fast as the hardware allows. Strategy selection considers both source and destination drives, automatically tuning parallelism and I/O mode for the specific combination.

Features

  • Dual-drive strategy selection — considers both source and destination drives to choose optimal parallelism, I/O mode, and copier for each combination
  • Parallel small-file copies via CopyFileEx with tuned thread count (CPU-clamped for local drives, capped at 3 for USB)
  • Unbuffered I/O (COPY_FILE_NO_BUFFERING) for large files (>100 MB) on local-to-local transfers to bypass OS cache (disabled when either drive is USB)
  • USB-optimized pipeline with double-buffered async reads, 1 MB buffers, SequentialScan, and destination pre-allocation — with automatic HDD vs SSD/flash sub-strategies triggered by either source or destination being USB
  • Instant same-volume moves via metadata rename
  • Real-time progress with transfer speed, labeled Elapsed/ETA columns, and per-file tracking (Spectre.Console)
  • Strategy display — shows the selected move strategy and rationale at startup
  • Global exception handler — unhandled exceptions written to stderr for diagnostics
  • Instant cancellation via Ctrl+C (kernel-level pbCancel flag)
  • CLI help — auto-generated --help with all options via Spectre.Console.Cli
  • Completion report — summary of moved/skipped/failed files, optionally saved to file
  • Merge into existing destination — silently merges instead of failing; existing files skipped by default
  • Single-file release — one 20 MB self-contained .exe, no runtime required

Benchmarks

All benchmarks are enforced by the test suite — the build fails if FastMove is not faster. Median of 3 runs after warmup, Windows 10 x64.

Scenario FastMove Baseline Speedup
200 x 64 KB files vs File.Copy 347 ms 536 ms 1.5x
200 x 64 KB files vs XCOPY 278 ms 953 ms 3.4x
200 x 64 KB files vs ROBOCOPY 236 ms 381 ms 1.6x
1 x 50 MB file vs 4 KB stream 51 ms 216 ms 4.2x
1 x 50 MB file vs XCOPY 27 ms 172 ms 6.4x
1 x 50 MB file vs ROBOCOPY 28 ms 114 ms 4.1x
Mixed (85 files) vs File.Copy 159 ms 188 ms 1.2x
Mixed (85 files) vs XCOPY 1254 ms 2566 ms 2.0x
Mixed (85 files) vs ROBOCOPY 152 ms 1784 ms 11.7x
Same-volume rename (100 files) 2 ms 916 ms 378x

Run benchmarks yourself:

dotnet test FastMove.Tests --filter "Category=Benchmark"

Usage

fastmove [options] <source> <destination>

Options:

Short Long Description
-s --skip <ATTRIBUTES> Skip files/directories with specified attributes. Accepts any FileAttributes enum name(s) (e.g. Hidden, System, ReadOnly). Can be repeated (--skip Hidden --skip System) or comma-separated (--skip Hidden,System).
--no-ignore-inaccessible Surface access-denied errors instead of silently skipping inaccessible files/directories (default is to ignore them).
-t --terminate-on-error Stop immediately on first error instead of continuing
-p --preserve-source Keep source directory after copying (copy mode)
-o --overwrite Overwrite existing files at destination instead of skipping
-r --report <PATH> Save completion report to file
-h --help Show help information
-v --version Show version information

Default behavior:

  • Continues on error (use --terminate-on-error to stop on first failure)
  • Source is preserved when any files fail to copy; deleted only on full success
  • Existing files at destination are skipped (use --overwrite to replace them)
  • Merges into an existing destination directory silently

Examples:

# Move a folder to another drive
fastmove D:\Projects\old-data E:\Archive\old-data

# Move within the same drive (instant rename)
fastmove C:\Users\me\Downloads\dataset C:\Data\dataset

# Skip hidden and system files (repeated flags)
fastmove --skip Hidden --skip System D:\Data E:\Backup

# Skip hidden and system files (comma-separated)
fastmove --skip Hidden,System D:\Data E:\Backup

# Skip read-only files
fastmove --skip ReadOnly D:\Data E:\Backup

# Surface access-denied errors instead of silently skipping
fastmove --no-ignore-inaccessible D:\Data E:\Backup

# Copy mode (preserve source)
fastmove --preserve-source D:\Data E:\Backup

# Overwrite existing files and save report
fastmove --overwrite --report report.txt D:\Data E:\Backup

# Stop on first error
fastmove --terminate-on-error D:\Data E:\Backup

Cancel: Press Ctrl+C at any time. The source directory is preserved on cancellation.

Build

Requires .NET 8 SDK.

# Debug build
dotnet build FastMove

# Optimized release (R2R, trimmed, single-file)
dotnet publish FastMove -c Release -o ./publish

# Run tests
dotnet test FastMove.Tests

The release build includes ReadyToRun pre-compilation, partial trimming, TieredPGO, and speed-optimized codegen.

How It Works

Same-volume moves: metadata rename

When the source and destination are on the same drive, there is nothing to copy. FastMove calls Directory.Move, which is a metadata-only rename operation at the filesystem level — the file contents never move. This is why same-volume moves complete in ~2 ms regardless of data size.

Cross-volume moves: CopyFileEx with parallel I/O

For cross-volume transfers, FastMove calls the Win32 CopyFileEx1 function directly via P/Invoke rather than using .NET's File.Copy. This provides three advantages:

  1. Progress callbacks. CopyFileEx accepts a CopyProgressRoutine2 callback that fires as each chunk is transferred, enabling real-time progress without polling.

  2. Instant cancellation. The pbCancel parameter is a pointer to a flag that CopyFileEx checks between chunks. Setting it to TRUE from any thread causes the copy to abort immediately — no waiting for the current buffer to flush.

  3. Unbuffered I/O for large files. The COPY_FILE_NO_BUFFERING flag performs the copy using unbuffered I/O, "bypassing system I/O cache resources"1. This avoids polluting the OS file cache3 when moving large files (>100 MB) that won't be read again soon. Small files are copied with buffering enabled, since the overhead of cache-aligned sector reads outweighs the benefit at small sizes.

For local-to-local transfers, files are partitioned at the 100 MB threshold: small files are copied in parallel using Parallel.ForEachAsync (thread count clamped to [2, ProcessorCount, 16]), while large files are copied sequentially with COPY_FILE_NO_BUFFERING to saturate disk bandwidth without contention. When either drive is USB, parallelism is capped at 3 and unbuffered I/O is disabled (see below).

USB/removable drive optimization

FastMove detects USB/removable drives on both source and destination using a two-tier approach: DriveInfo.DriveType == Removable catches USB flash drives, then a PowerShell subprocess queries Get-PhysicalDisk for BusType to detect USB hard drives and SSDs that report as Fixed. (PowerShell is used instead of System.Management because the managed WMI library is incompatible with IL trimming.)

When either drive is USB, FastMove adjusts its strategy:

  • Reduced parallelism (3 threads vs up to 16). USB is a shared bus — too many concurrent transfers cause contention rather than throughput gains.
  • No unbuffered I/O. COPY_FILE_NO_BUFFERING bypasses the OS page cache, which hurts USB throughput. USB drives rely on OS write-back coalescing since most use "Quick Removal" policy with no device-level write caching.

When the source is USB, FastMove additionally switches to a dedicated UsbFileCopier:

  • Buffered reads with SequentialScan to maximize OS read-ahead prefetching on the USB bus, instead of NO_BUFFERING which would disable it.
  • 1 MB buffers instead of CopyFileEx's ~64 KB default. USB drives benefit from larger transfer units that amortize per-request overhead.
  • Double-buffered async pipelining using the .NET RandomAccess API4: ReadAsync fills buffer A while WriteAsync drains buffer B, then they swap. This keeps the USB bus continuously saturated rather than alternating idle/active.
  • Destination pre-allocation via RandomAccess.SetLength before writing, which reduces fragmentation5 by letting NTFS allocate contiguous extents up front.

USB HDD sub-strategy

USB spinning hard drives suffer 2–10x throughput loss when multiple large files are copied in parallel, because the mechanical head must seek between concurrent streams. FastMove detects USB HDDs via Get-PhysicalDisk MediaType (HDD = 3, Unspecified = 0 treated conservatively as HDD, SSD = 4). When either drive is a USB HDD, the sub-strategy applies:

  • Large files (≥100 MB): copied serially (1-way) to avoid head thrashing
  • Small files (<100 MB): keep 3-way parallelism (per-file seek overhead is negligible)

When USB Flash/SSD is involved but no HDD, all files use flat 3-way parallelism since there is no mechanical head.

Cancellation architecture

FastMove uses a dual cancellation mechanism. The pbCancel integer flag (shared by reference across all CopyFileEx calls) provides kernel-level cancellation that aborts the current copy mid-transfer. A standard CancellationToken is passed to Parallel.ForEachAsync to prevent new files from starting. When the user presses Ctrl+C, both are triggered simultaneously, so the response is effectively instant — no waiting for a multi-megabyte buffer to finish flushing.

Progress tracking

A lock-free ProgressTracker uses Interlocked operations for all counters (bytes copied, files completed, current file progress). The Spectre.Console UI thread polls at 20 Hz to update the two progress bars (overall and current file) without any locking contention with the copy threads. Progress descriptions show the relative path from the source root (padded to a fixed 48-character width to prevent column dancing), with labeled "Elapsed" and "ETA" time columns in distinct colors.

License

MIT

Footnotes

  1. CopyFileExA function — Microsoft Learn 2

  2. CopyProgressRoutine callback — Microsoft Learn

  3. File Buffering — Microsoft Learn

  4. RandomAccess Class — Microsoft Learn

  5. Raymond Chen, "How can I pre-allocate a file?" — The Old New Thing

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages