Inspired by Dgraph's mmap file implementation in ristretto.
A file-backed memory map exposes the kernel's view of an inode as a &[u8]/&mut [u8]. That makes it easy to reach for, but it also means UB the moment another actor truncates, unlinks, or rewrites the file out from under the mapping — SIGBUS on Unix, mapping detachment on Windows, silent torn reads in either. fmmap raises a safe API over memmapix by treating those concerns as first-class:
- Auto-acquired advisory lock on every constructor — exclusive on writable maps, shared on read-only / COW maps. Aliased writable mappings of the same file (and mut-then-COW) are rejected up front.
- Best-effort path-reuse mitigation on deletion. Identity is captured at open and re-checked before every unlink so a file someone else has swapped in at the path won't be silently deleted. POSIX uses
(st_dev, st_ino); Windows uses(volumeSerial, fileIndex)fromGetFileInformationByHandle(viawindows-sys, no nightly required). This is not an absolute guarantee — see the path-reuse limitations below. - Pre-validated mapping ranges. Constructors reject
offset/lenoverflow, ranges past EOF, and effective lengths >isize::MAXbefore any destructiveset_lenruns, so an invalidOptionsnever zeroes or extends an existing file. - Crash-durable unlink. The parent directory is pinned by a handle opened before
remove_file, then fsynced through that same handle. Failed-fsync retries fsync the same handle (not a freshly-opened parent), so a parent rename between unlink and fsync can't direct the durability to the wrong inode. - Reentrant-safe lock methods.
LockFileExdeadlocks on the same Windows handle;lock/lock_sharedshort-circuit when the desired state is already held. The lock methods take&mut selfso single-owner serialization is enforced by the borrow checker. - Poison-safe truncate / freeze. A failed truncate marks the wrapper poisoned; subsequent reads return
&[]and writes/flushes/freezes returnErrrather than handing back an anonymous-mapped placeholder pretending to be the original file.
std plus tokio and smol are first-class. The async surface is built from the same set of macros, so adding a new runtime is small and mechanical — see fmmap/src/disk/{tokio,smol}_impl.rs.
Identity-checked deletion is built on the strongest atomic primitives each platform exposes; what's left is a small, documented set of irreducible races.
POSIX: probe + unlink + parent fsync are all bound to the same parent fd via rustix's fstatat + unlinkat. A parent rename mid-operation can't direct the unlink or fsync to a different directory than the one we verified. The original file's open-file description is held alive (via fcntl(F_DUPFD_CLOEXEC) or, in the tokio wrapper, tokio::fs::File::into_std()) across probe + unlink, so the kernel cannot recycle (dev, ino) to a fresh file in the window. Identity capture itself is allocation-free (fstat on a BorrowedFd), so EMFILE has no path to defeating the identity check.
Windows: probe and unlink are bound to a single handle. The handle is opened with DELETE | FILE_SHARE_* and FILE_FLAG_OPEN_REPARSE_POINT; we re-verify identity and refuse reparse points on that handle, then issue SetFileInformationByHandle(FileDispositionInfoEx) with POSIX_SEMANTICS | IGNORE_READONLY_ATTRIBUTE. Older Windows / FAT32 fall back to FileDispositionInfo after a ReOpenFile widens access to clear FILE_ATTRIBUTE_READONLY (using FILE_ATTRIBUTE_NORMAL as the cleared-state sentinel — Windows treats 0 as "no change"). Identity is captured directly via GetFileInformationByHandle on a borrowed HANDLE — no DuplicateHandle, no fd alloc.
API contract: explicit remove() (and drop_remove()) only returns Ok if fmmap itself observed the unlink succeed in the parent it then fsynced. NotFound from the probe or unlink is never converted into a durable-success retry — the wrapper stays in NeedsUnlink and surfaces the error, even when the inode's nlink has dropped to 0 (which can't distinguish "unlink in our parent" from "external rename + unlink elsewhere"). Drop's best-effort cleanup still fsyncs the parent in the common case, but the API doesn't promise durability we can't verify.
- One-syscall TOCTOU on POSIX. Between
fstatatandunlinkat— both bound to the same parent fd — there's still a single-syscall window where the entry could be replaced. Closing this needs an inode-boundunlinkatprimitive POSIX doesn't expose. The window is dramatically narrower than the handle-drop-to-retry window the identity check does close, but it's not zero. - External rename + unlink elsewhere. A concurrent actor can rename our file into a different directory and unlink it there. The inode's
nlinkdrops to 0 but our parent's fsync doesn't commit their unlink. fmmap detects this only as "the file is gone" and surfacesNotFound; under that scenario, callers who need crash-durability should serialize external mutations orfsyncthe relevant parents themselves. - Smol consuming
drop_remove(self)under EMFILE. smol'sasync-fs::Fileexposes nointo_std(), so the inode pin is afcntl_dupfd_cloexecof the underlying fd. Under fd pressure the dup fails,drop_removereturnsErrdeterministically (no hidden Drop-time retry), and the file remains on disk. Callers can recover viastd::fs::remove_file(path)directly orAsyncMmapFileMut::remove(&mut self)which preservesselffor an explicit retry. Tokio'sinto_std()allocates no fd so this limitation doesn't apply on tokio.
If your threat model includes an active local adversary, do not rely on identity-checked delete for safety — perform the cleanup yourself with whatever atomic primitives your platform provides.
- file-backed memory maps with auto-locked construction
- read-only / copy-on-write / mutable / executable maps
- identity-checked deletion bound to a single kernel-verified handle (POSIX
fstatat+unlinkaton a parent fd; WindowsSetFileInformationByHandle(FileDispositionInfoEx)on aDELETE | FILE_SHARE_DELETEhandle); see Design for residual races - inode pin across probe + unlink (POSIX
F_DUPFD_CLOEXECor tokiointo_std) — defends against(dev, ino)recycling on tmpfs / small-id filesystems - crash-durable unlink with pre-opened parent fsync (same handle reused on retry)
- symlink / reparse-point refusal at the same syscall as the identity probe (POSIX
AT_SYMLINK_NOFOLLOW, WindowsFILE_FLAG_OPEN_REPARSE_POINT) - readonly-file delete on Windows (
FileDispositionInfoExwithIGNORE_READONLY_ATTRIBUTE, legacyFileDispositionInfofallback for pre-1607) - pre-validated mapping ranges (rejects past-EOF and
> isize::MAXbefore any destructiveset_len) - poison-safe
truncate/freeze/freeze_exec - synchronous and asynchronous flushing
- reader / writer adapters with byteorder + seek
- dozens of file I/O util functions
- stack support (
MAP_STACKon Unix) - tokio
- smol
fmmap requires Rust 1.75 or later.
-
std
[dependencies] fmmap = "0.5"
-
[dependencies] fmmap = { version = "0.5", default-features = false, features = ["tokio"] }
-
[dependencies] fmmap = { version = "0.5", default-features = false, features = ["smol"] }
The sync feature is on by default.
This crate is 100% documented, see docs.rs for examples.
Licensed under either of Apache License, Version 2.0 or MIT license at your option.Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.