async_file

Asynchronous file I/O operations with priority handling.

async_file provides a simple yet powerful API for performing asynchronous file operations in Rust. It closely follows the standard library's file API design while adding async support and priority-based scheduling.

Features

• Async Operations: All file operations are asynchronous, allowing for non-blocking I/O
• Priority Scheduling: Every operation accepts a priority parameter for fine-grained control
• Memory Safety: Uses an opaque Data type to safely handle OS-managed memory allocations
• Platform Agnostic: Backend-agnostic API with a default std implementation

Quick Start

use async_file::{File, Priority};

// Open a file with unit test priority
let file = File::open("/dev/zero", Priority::unit_test()).await?;

// Read up to 1KB of data
let data = file.read(1024, Priority::unit_test()).await?;
println!("Read {} bytes", data.len());

Architecture Overview

Opaque Type Design

The library uses opaque wrapper types that hide platform-specific implementations:

• File: Wraps platform file handles behind a unified async interface
• Data: Encapsulates OS-managed memory buffers for safe async I/O
• Metadata: Provides file information in a platform-agnostic way
• Error: Wraps platform-specific error types

This design ensures API stability while allowing platform-specific optimizations.

Single Operation Constraint

Important: Only one operation may be in-flight at a time per file handle.

This constraint:

• Prevents race conditions on file position
• Simplifies the implementation
• Avoids many classes of concurrency bugs
• Matches typical file I/O patterns

Attempting concurrent operations on the same file handle will result in undefined behavior.

Memory Management Strategy

The library uses an opaque Data type instead of user-provided buffers. This design:

• Prevents use-after-free bugs: If an async operation is cancelled (by dropping the future), the OS might still write to the buffer. OS-managed allocation prevents this.
• Enables platform optimizations: Different platforms can use their optimal memory allocation strategies.
• Simplifies the API: Users don't need to manage buffer lifetimes across await points.

Common Usage Patterns

Reading a File Completely

use async_file::{File, Priority};

// For small files, use read_all()
let file = File::open("config.txt", Priority::highest_async()).await?;
let contents = file.read_all(Priority::highest_async()).await?;

// Convert to String if needed
let text = String::from_utf8(contents.into_boxed_slice().into_vec())
    .expect("Invalid UTF-8");

Sequential Reading with Seeking

use async_file::{File, Priority};
use std::io::SeekFrom;

let mut file = File::open("/dev/zero", Priority::unit_test()).await?;

// Read header (first 128 bytes)
let header = file.read(128, Priority::unit_test()).await?;

// Skip to data section at byte 1024
file.seek(SeekFrom::Start(1024), Priority::unit_test()).await?;

// Read data
let data = file.read(4096, Priority::unit_test()).await?;

Checking File Existence Before Opening

use async_file::{exists, File, Priority};

let path = "important.dat";

if exists(path, Priority::unit_test()).await {
    let file = File::open(path, Priority::highest_async()).await?;
    // Process file...
} else {
    eprintln!("File not found: {}", path);
}

Priority-Based Operations

use async_file::{File, Priority};

// Critical system file - use highest priority
let system_file = File::open("/critical/system.conf",
    Priority::highest_async()).await?;

// Background logging - use low priority
// For other priority levels, use Priority::new()
// Priority::new(0.2) for low priority tasks

// User-facing operation - use high priority
// Priority::new(0.8) for high priority tasks

// Unit tests - use dedicated test priority
let test_file = File::open("test_fixture.txt",
    Priority::unit_test()).await?;

API Overview

File Operations

use async_file::{File, Priority};
use std::io::SeekFrom;

// Open a file
let mut file = File::open("/path/to/file", Priority::unit_test()).await?;

// Read data
let data = file.read(1024, Priority::unit_test()).await?;

// Seek to position
let pos = file.seek(SeekFrom::Start(100), Priority::unit_test()).await?;

// Get metadata
let metadata = file.metadata(Priority::unit_test()).await?;
println!("File size: {} bytes", metadata.len());

// Read entire file
let contents = file.read_all(Priority::unit_test()).await?;

Memory Management

The Data type provides safe access to OS-managed memory:

let data = file.read(100, Priority::unit_test()).await?;

// Access as a slice
let bytes: &[u8] = data.as_ref();

// Convert to owned data (may require copying)
let boxed: Box<[u8]> = data.into_boxed_slice();

Platform Support

• Unix/Linux/macOS: Uses blocking crate to run std::fs operations in a thread pool
• WASM: Uses web fetch API for remote file access
  • Origin detection (in order of priority):
    1. Configured origin via set_default_origin() (if set)
    2. window.location.origin (browser main thread)
    3. self.origin (web workers)
    4. Panics if no origin can be determined (e.g., Node.js without configuration)
  • Important: Servers must not compress responses. Compressed content causes a CompressedResponse error because the Content-Length reflects compressed size (not actual file size), breaking metadata and range requests. Configure servers to disable compression or use Accept-Encoding: identity.
• Windows: Same as Unix implementation using blocking crate

Utility Functions

Checking File Existence

use async_file::{exists, Priority};

// Check if a file exists
if exists("/path/to/file", Priority::unit_test()).await {
    println!("File exists");
} else {
    println!("File not found");
}

Setting Default Origin for WASM

In WASM environments, files are fetched from remote URLs rather than accessed from a local filesystem. Use set_default_origin to set the base URL for these fetch operations.

use async_file::set_default_origin;

// Set origin for WASM file fetching
set_default_origin("https://cdn.example.com/assets");

// On non-WASM platforms, this is a no-op

When to use: Call this function at application startup when:

• Running in Node.js or other non-browser WASM environments where automatic detection fails
• You need to override the automatically detected origin (e.g., to fetch files from a CDN)
• You want to explicitly control the base URL for file operations

Priority System

All operations require a priority parameter from the priority crate for scheduling control:

use async_file::Priority;

// Different priority levels
let high_priority = Priority::highest_async();
let test_priority = Priority::unit_test();

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.