中文文档

FluxCache is a C++20 distributed cache file system prototype inspired by Alluxio and GooseFS.

The project is intentionally being built around a narrow MVP first:

• single Master
• single Worker
• LocalFS as the first UFS
• page cache in Memory first
• write-through writes
• mtime-based cache validation in Phase 1

Current focus: freeze the Phase 1 contract and make the first end-to-end read/write path verifiable.

The repository should be read as:

• a stable architecture direction
• an actively evolving implementation roadmap
• not a claim that all planned features are already production ready

Client
  |  gRPC
  v
Single Master (MountTable + InodeTree + WorkerManager + HashRing)
  |  gRPC
  v
Single Worker (Memory Tier + PageStore)
  |
  v
Under File System (LocalFS first)

• Client
  • asks Master for file metadata
  • computes BlockId locally
  • routes reads/writes to Worker
• Master
  • owns namespace and inode metadata
  • resolves mount paths
  • exposes worker list and ring version
• Worker
  • serves page reads/writes
  • caches pages locally
  • falls back to UFS on cache miss
• UFS
  • remains the durable source of truth

• InodeId is the file identity.
• BlockId = (InodeId, BlockIndex) is the routing identity.
• PageId = {BlockId, page_index} is the cache identity.
• Master does not store Block -> Worker locations in Phase 1.
• Worker does not own file path metadata.
• Cache freshness is checked with ufs_mtime_ms in Phase 1.

1. Client calls GetFileInfo.
2. Master returns inode_id, size, block_size, ufs_mtime_ms, ring_version, workers, ufs_uri, and ufs_path.
3. Client computes BlockId and page_indices locally.
4. Client sends ReadPages to Worker.
5. Worker checks PageStore.
6. On miss or stale cache, Worker reads from UFS and refills cache.

1. Client calls CreateFile or GetFileInfo.
2. Client computes target blocks/pages locally.
3. Client sends WritePages to Worker.
4. Worker writes to UFS first.
5. Worker updates cache only after UFS write succeeds.
6. Client calls CompleteFile so Master updates metadata.

The authoritative roadmap lives in issues/index.md.

High-level phases:

• Phase A: contract freeze
• Phase B: metadata-plane closure
• Phase C: data-plane closure
• Phase D: recovery and deletion semantics
• Phase E: multi-worker and transport hardening
• Phase F: multi-tier cache and eviction
• Phase G: access surfaces (CLI, SDK, FUSE)
• Phase H: backend expansion (S3, HDFS)
• Phase I: HA and resilience
• Phase J: performance, observability, and quality track

These are still roadmap items, not current stable core claims:

• SSD/HDD tiers
• MetaStore-based worker recovery
• FUSE access
• C++ SDK beyond file-path MVP
• S3 / HDFS backends
• HA journal / Raft (suspended; prototype behind FLUXCACHE_ENABLE_RAFT=OFF)
• advanced resilience and degradation
• production observability

• C++20 compiler
• CMake 3.20+
• Protobuf + gRPC
• RocksDB
• yaml-cpp
• S3/MinIO: vcpkg + minio-cpp (required)

Optional later-stage dependencies:

• FUSE3
• HDFS client libraries
• Prometheus C++ client

S3 UFS (minio-cpp) is required. The simplest way is to use the build script:

./build.sh

This uses the project's vcpkg/ (or VCPKG_ROOT if set), bootstraps vcpkg if needed, and builds. Optional: ./build.sh [build_dir] [cmake_args...], e.g. ./build.sh build -DFLUXCACHE_ENABLE_FUSE=ON.

Or manually with vcpkg:

mkdir -p build && cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE=[path-to-vcpkg]/scripts/buildsystems/vcpkg.cmake
cmake --build .
# For IDE: ln -sf "$(pwd)/compile_commands.json" ../compile_commands.json

IDE IntelliSense (clangd / C++ extension): Run ./build.sh once (or create the symlink above after manual cmake). This lets Cursor/VSCode resolve includes like worker/page/page_store.h and jump to standard library headers.

cd build
ctest --output-on-failure

When changing the roadmap or implementation, keep these files consistent:

• README.md
• README.zh-CN.md
• issues/index.md
• plan/issue-status.md
• relevant docs in design/

MIT License