Skip to content

Latest commit

 

History

History
1815 lines (1502 loc) · 72.9 KB

File metadata and controls

1815 lines (1502 loc) · 72.9 KB

Ferripfs: Kubo to Rust Port Plan

This document is the controlling specification for the ferripfs project.


Implementation Status

Phase Name Status Compliance Tests Kubo Tests Ported Notes
1 Project Foundation COMPLETE 10/10 N/A Workspace, licenses, docs, CLI skeleton
2 Configuration and Repository COMPLETE 27/27 17/25 files Config, repo, and migration tests ported
3 Blockstore and Content Addressing INCOMPLETE 15/15 0/5+ files Implementation done, Go tests NOT ported
4 UnixFS and File Operations INCOMPLETE 27/27 0/6+ files Implementation done, Go tests NOT ported
5 Pinning System INCOMPLETE 19/19 0/3+ files Implementation done, Go tests NOT ported
6 DAG Operations INCOMPLETE 13/13 0/14+ files Implementation done, Go tests NOT ported
7 Networking (libp2p) INCOMPLETE 12/12 0/3 files Implementation done, Go tests NOT ported
8 DHT and Routing NOT STARTED 0/16 0/4+ files Kademlia, content routing
9 Bitswap and Content Exchange NOT STARTED 0/12 0/5+ files Block exchange protocol
10 IPNS and Key Management NOT STARTED 0/23 0/6+ files IPNS records, key gen
11 MFS (Mutable File System) NOT STARTED 0/20 0/3+ files files subcommand
12 PubSub NOT STARTED 0/10 0/4+ files GossipSub/FloodSub
13 HTTP Gateway NOT STARTED 0/15 0/4+ files Path/subdomain gateway
14 HTTP RPC API NOT STARTED 0/10 0/4+ files JSON-RPC API server
15 Remaining Commands NOT STARTED 0/varies 0/10+ files Full CLI parity

CRITICAL: A phase is NOT COMPLETE until:

  1. Implementation passes all compliance tests
  2. ALL relevant Kubo Go tests have been ported and pass
  3. Port attribution headers reference original Go test files

This requirement applies to ALL phases (2-15). See "Kubo Test Files to Port (Detailed)" section below for the complete list of Go tests that must be ported for each phase.

Crate Structure

ferripfs/
├── bin/ferripfs/           # Main CLI binary
├── crates/
│   ├── ferripfs-blockstore/   # Block storage, CID handling
│   ├── ferripfs-config/       # Configuration types
│   ├── ferripfs-dag/          # DAG/IPLD operations
│   ├── ferripfs-network/      # libp2p networking (Phase 7)
│   ├── ferripfs-pinning/      # Pinning system
│   ├── ferripfs-repo/         # Repository management
│   ├── ferripfs-test-harness/ # Testing infrastructure
│   ├── ferripfs-tests/        # Integration tests
│   │   └── tests/
│   │       ├── phase1_compliance.rs   # Phase 1 compliance tests
│   │       ├── phase2_compliance.rs   # Phase 2 compliance tests
│   │       ├── ...                    # phaseN_compliance.rs for each phase
│   │       └── kubo_ported/           # PORTED KUBO GO TESTS (REQUIRED)
│   │           ├── phase2_config_test.rs
│   │           ├── phase2_repo_test.rs
│   │           ├── phase3_blockstore_test.rs
│   │           ├── phase4_unixfs_test.rs
│   │           ├── phase5_pinning_test.rs
│   │           ├── phase6_dag_test.rs
│   │           ├── phase7_network_test.rs
│   │           ├── phase8_dht_test.rs
│   │           ├── phase9_bitswap_test.rs
│   │           ├── phase10_ipns_test.rs
│   │           ├── phase11_mfs_test.rs
│   │           ├── phase12_pubsub_test.rs
│   │           ├── phase13_gateway_test.rs
│   │           ├── phase14_rpc_test.rs
│   │           └── phase15_commands_test.rs
│   └── ferripfs-unixfs/       # UnixFS implementation

Project Overview

Goal: Port Kubo (IPFS Go implementation) v0.39.0 to Rust with full feature and CLI command parity.

Project Name: ferripfs

Source: /home/dpp/tmp/rustipfs/kubo (tag v0.39.0)

License: Dual MIT/Apache-2.0 (matching Kubo)

Approach: Port fresh from Kubo for maximum fidelity; use only low-level ecosystem crates (cid, multihash, libp2p)

Principles:

  • Ethical, transparent port with explicit attribution
  • Every ported file references its Kubo source
  • Feature-by-feature and command-by-command verifiable parity
  • Rust library + CLI binary
  • 85% minimum test coverage per phase
  • Human-readable compliance reports from integration tests
  • Prefer existing open source implementations (see Library-First Development below)

Library-First Development Policy

Core Principle

Never write a module or functionality if there is an existing open source implementation available.

The Rust ecosystem has mature crates for most IPFS-related functionality. Before writing any code, search for and evaluate existing crates that provide the needed functionality.

Required Behavior

  1. Search Before Writing: Before implementing any module, search crates.io and GitHub for existing implementations. Evaluate:

    • rust-ipfs ecosystem crates
    • libp2p crates for networking
    • cid, multihash, multiaddr for content addressing
    • prost with pre-generated code or quick-protobuf for protobuf
    • Any other well-maintained crates that provide the needed functionality
  2. Stop on Missing Dependencies: If a build or runtime dependency is missing (e.g., protoc not installed, system library unavailable), STOP immediately and prompt the user to install the required dependency. Do not attempt workarounds like:

    • Writing hand-coded alternatives
    • Generating code manually
    • Using different approaches to avoid the dependency
  3. Permission Required for Custom Code: If custom code must be written instead of using an existing library, ask the user for explicit permission with a complete justification that includes:

    • What libraries were evaluated and why they are unsuitable
    • What specific functionality is missing from existing crates
    • Why the custom implementation is necessary
    • The scope and complexity of the custom code

Example Scenarios

Scenario: Protobuf compilation requires protoc

  • WRONG: Write hand-coded protobuf structures to avoid the dependency
  • RIGHT: Stop and tell the user: "Building ferripfs-unixfs requires protoc. Please install it with: sudo apt-get install protobuf-compiler"

Scenario: Need UnixFS chunking

  • FIRST: Search for existing crates (ipfs-unixfs, rust-ipfs modules, etc.)
  • IF NONE SUITABLE: Ask user: "I could not find an existing Rust crate for UnixFS chunking that matches Kubo's implementation. The closest is X but it lacks Y. May I implement custom chunking code?"

Scenario: Need CID handling

  • WRONG: Implement CID parsing from scratch
  • RIGHT: Use the cid crate from the rust-ipfs ecosystem

Approved Dependencies

The following crates are pre-approved for use without further justification:

  • cid, multihash, multihash-codetable, multiaddr - Content addressing
  • libp2p and its sub-crates - Networking
  • prost, prost-build - Protobuf (requires protoc)
  • serde, serde_json - Serialization
  • tokio - Async runtime
  • clap - CLI parsing
  • thiserror, anyhow - Error handling
  • Standard Rust ecosystem crates for common tasks

Testing Framework Requirements

Test Coverage Standards

Each phase must achieve:

  • 85% line coverage measured by cargo-tarpaulin or llvm-cov
  • 100% coverage of public API surface
  • Integration tests for every requirement
  • Ported Kubo Go tests for all relevant functionality

Kubo Test Porting Requirement

For each phase, the relevant Go tests from Kubo must be identified, ported to Rust, and included as part of the phase's test suite.

Process

  1. Identify Relevant Kubo Tests: Before implementing a phase, locate all Go test files in Kubo that test the functionality being ported. Key locations include:

    • kubo/test/ - Integration and CLI tests
    • kubo/core/commands/*_test.go - Command tests
    • boxo/*/test/ - Component tests (blockstore, unixfs, etc.)
    • Package-level *_test.go files adjacent to implementation
  2. Document Test Mapping: Create a test mapping document for each phase listing:

    • Original Go test file path
    • Test function names
    • Corresponding Rust test location
    • Any behavioral differences or limitations
  3. Port Tests Faithfully: Tests should be ported to maintain:

    • Same test inputs and expected outputs
    • Same edge cases and error conditions
    • Same behavioral assertions
    • Comments referencing the original Go test
  4. Test File Naming Convention: Ported tests should follow:

    tests/kubo_ported/
    ├── phase1_tests.rs
    ├── phase2_config_test.rs
    ├── phase2_repo_test.rs
    └── ...
    
  5. Attribution in Ported Tests: Each ported test file must include:

    // Ported from: kubo/path/to/original_test.go
    // Kubo version: v0.39.0
    // Original test: TestFunctionName
    //
    // Original work: Copyright (c) Protocol Labs, Inc.
    // Port: Copyright (c) 2026 ferripfs contributors
    // SPDX-License-Identifier: MIT OR Apache-2.0

Kubo Test Files to Port (Detailed)

The following lists the specific Go test files in Kubo v0.39.0 that MUST be ported for each phase. A phase is not complete until all listed tests are ported.


Phase 2: Configuration and Repository (25 test files, 150+ functions)

Config Tests:

File Test Functions Port Status
kubo/config/api_test.go TestConvertAuthSecret PORTEDphase2_config.rs::test_convert_auth_secret
kubo/config/autoconf_test.go TestAutoConfDefaults, TestAutoConfProfile, TestInitWithAutoValues DEFERRED (autoconf not implemented)
kubo/config/bootstrap_peers_test.go TestBootstrapPeerStrings PORTEDphase2_config.rs::test_bootstrap_peer_strings
kubo/config/config_test.go TestClone, TestReflectToMap, TestCheckKey PORTEDphase2_config.rs::test_clone, test_check_key
kubo/config/import_test.go 10 validation tests (HAMTFanout, CidVersion, UnixFSFileMaxLinks, etc.) DEFERRED (validation API not implemented)
kubo/config/init_test.go TestCreateIdentity, TestCreateIdentityOptions DEFERRED (CreateIdentity in ferripfs-repo, not config)
kubo/config/migration_test.go TestMigrationDecode PORTEDphase2_config.rs::test_migration_decode
kubo/config/provide_test.go TestParseProvideStrategy, TestValidateProvideConfig_*, TestShouldProvideForStrategy DEFERRED (provide strategy not implemented)
kubo/config/routing_test.go TestRouterParameters, TestMethods PORTEDphase2_config.rs::test_router_parameters_roundtrip
kubo/config/serialize/serialize_test.go TestConfig DEFERRED (file doesn't exist in kubo)
kubo/config/types_test.go 10 type tests (OptionalDuration, Strings, Flag, Priority, Integer, String, Bytes) PORTEDphase2_config.rs::test_optional_*, test_strings_*, test_flag_*, test_priority_*

Repository Tests:

File Test Functions Port Status
kubo/repo/common/common_test.go 4 map merge tests PORTEDphase2_repo.rs::test_map_merge_deep_*
kubo/repo/fsrepo/config_test.go 4 datastore config tests DEFERRED (datastore plugin system not implemented)
kubo/repo/fsrepo/fsrepo_test.go 5 repo lifecycle tests PORTEDphase2_repo.rs::test_init_*, test_can_manage_*, test_datastore_*, test_open_*

Migration Tests:

File Test Functions Port Status
kubo/repo/fsrepo/migrations/atomicfile/atomicfile_test.go 9 atomic file tests PORTEDphase2_migrations.rs::test_atomic_file_*
kubo/repo/fsrepo/migrations/embedded_test.go 3 embedded migration tests PORTEDphase2_migrations.rs::test_*_embedded_migration*
kubo/repo/fsrepo/migrations/fetch_test.go TestGetDistPath, TestMultiFetcher PORTEDphase2_migrations.rs::test_get_dist_path
kubo/repo/fsrepo/migrations/ipfsdir_test.go TestExpandHome, TestIpfsDir, TestCheckIpfsDir, TestRepoVersion PORTEDphase2_migrations.rs::test_expand_home_*, test_ipfs_dir_*, test_check_ipfs_dir_*, test_repo_version*
kubo/repo/fsrepo/migrations/migrations_test.go TestFindMigrations, TestMigrationName, TestExeName, TestNeedMigration PORTEDphase2_migrations.rs::test_find_migrations*, test_migration_name, test_exe_name, test_need_migration
kubo/repo/fsrepo/migrations/versions_test.go TestParseSemver, TestCompareVersions, TestSortVersions PORTEDphase2_migrations.rs::test_parse_semver, test_compare_versions, test_sort_versions, test_parse_versions_list, test_find_latest*

Phase 3: Blockstore and Content Addressing

Note: Blockstore tests are in the boxo library (github.com/ipfs/boxo), not in kubo repo. Must be obtained from boxo v0.35.2.

Source Test Functions Port Status
boxo/blockstore/blockstore_test.go Blockstore CRUD operations NOT PORTED
boxo/blockstore/bloom_cache_test.go Bloom filter caching tests NOT PORTED
boxo/blockstore/caching_test.go Caching blockstore tests NOT PORTED
go-cid library CID v0/v1 parsing and encoding tests NOT PORTED
go-multihash library Multihash encoding tests NOT PORTED

Phase 4: UnixFS and File Operations (2 test files in kubo, more in boxo)

Kubo Tests:

File Test Functions Port Status
kubo/core/coreunix/add_test.go TestAddMultipleGCLive, TestAddGCLive, TestAddWPosInfo, TestAddWPosInfoAndRawLeafs NOT PORTED
kubo/core/coreunix/metadata_test.go TestMetadata NOT PORTED

Boxo Tests (from boxo library):

Source Test Functions Port Status
boxo/chunker/splitting_test.go Size chunker tests NOT PORTED
boxo/chunker/rabin_test.go Rabin chunker tests NOT PORTED
boxo/ipld/unixfs/importer/*_test.go DAG builder tests NOT PORTED
boxo/ipld/unixfs/unixfs_test.go UnixFS encoding tests NOT PORTED

Phase 5: Pinning System (1 test file in kubo, more in boxo)

Kubo Tests:

File Test Functions Port Status
kubo/core/commands/pin/remotepin_test.go TestNormalizeEndpoint NOT PORTED

Boxo Tests (from boxo library):

Source Test Functions Port Status
boxo/pinning/pinner/pin_test.go Pin/unpin operations, recursive pins NOT PORTED
boxo/pinning/pinner/dspinner/*_test.go Datastore pinner tests NOT PORTED

Phase 6: DAG Operations (12 test files)

Command Tests:

File Test Functions Port Status
kubo/core/commands/cid_test.go CID command tests NOT PORTED
kubo/core/commands/commands_test.go Command structure tests NOT PORTED
kubo/core/commands/config_test.go Config command tests NOT PORTED
kubo/core/commands/dht_test.go DHT command tests NOT PORTED
kubo/core/commands/files_test.go Files command tests NOT PORTED
kubo/core/commands/get_test.go Get command tests NOT PORTED
kubo/core/commands/helptext_test.go Help text tests NOT PORTED
kubo/core/commands/repo_verify_test.go Repo verify tests NOT PORTED
kubo/core/commands/root_test.go Root command tests NOT PORTED
kubo/core/commands/cidbase_test.go CID base tests NOT PORTED
kubo/core/commands/env_test.go Environment tests NOT PORTED
kubo/core/commands/utils_test.go TestPathOrCidPath, TestValidatePinName NOT PORTED

Boxo Tests (from boxo library):

Source Test Functions Port Status
boxo/ipld/merkledag/*_test.go MerkleDAG tests NOT PORTED
boxo/ipld/car/*_test.go CAR format tests NOT PORTED

Phase 7: Networking (3 test files)
File Test Functions Port Status
kubo/core/node/libp2p/libp2p_test.go TestPrioritize NOT PORTED
kubo/core/node/libp2p/rcmgr_logging_test.go TestLoggingResourceManager NOT PORTED
kubo/core/node/libp2p/routingopt_test.go TestHttpAddrsFromConfig, TestDetermineCapabilities, TestEndpointCapabilitiesReadWriteLogic NOT PORTED

Phase 8: DHT and Routing

Kubo Tests:

File Test Functions Port Status
kubo/core/commands/dht_test.go DHT command tests NOT PORTED
kubo/routing/*_test.go Routing tests NOT PORTED

External Library Tests (go-libp2p-kad-dht):

Source Test Functions Port Status
go-libp2p-kad-dht/*_test.go Kademlia DHT tests NOT PORTED
go-libp2p-kad-dht/routing_test.go Routing table tests NOT PORTED

Phase 9: Bitswap and Content Exchange

Boxo Tests:

Source Test Functions Port Status
boxo/bitswap/bitswap_test.go Core bitswap tests NOT PORTED
boxo/bitswap/client/*_test.go Bitswap client tests NOT PORTED
boxo/bitswap/server/*_test.go Bitswap server tests NOT PORTED
boxo/bitswap/wantlist/*_test.go Wantlist tests NOT PORTED
boxo/bitswap/decision/*_test.go Decision engine tests NOT PORTED

Phase 10: IPNS and Key Management

Kubo Tests:

File Test Functions Port Status
kubo/core/commands/keystore_test.go Keystore command tests NOT PORTED
kubo/core/commands/name/*_test.go Name command tests NOT PORTED

Boxo Tests:

Source Test Functions Port Status
boxo/ipns/*_test.go IPNS record tests NOT PORTED
boxo/namesys/*_test.go Name system resolver tests NOT PORTED
boxo/namesys/publisher_test.go IPNS publisher tests NOT PORTED
boxo/keystore/*_test.go Key storage tests NOT PORTED

Phase 11: MFS (Mutable File System)

Kubo Tests:

File Test Functions Port Status
kubo/core/commands/files_test.go Files command tests NOT PORTED

Boxo Tests:

Source Test Functions Port Status
boxo/mfs/*_test.go MFS directory/file tests NOT PORTED
boxo/mfs/mfs_test.go Core MFS operations NOT PORTED

Phase 12: PubSub

External Library Tests (go-libp2p-pubsub):

Source Test Functions Port Status
go-libp2p-pubsub/*_test.go PubSub tests NOT PORTED
go-libp2p-pubsub/gossipsub_test.go GossipSub tests NOT PORTED
go-libp2p-pubsub/floodsub_test.go FloodSub tests NOT PORTED

Kubo Tests:

File Test Functions Port Status
kubo/core/commands/pubsub_test.go PubSub command tests NOT PORTED

Phase 13: HTTP Gateway

Boxo Tests:

Source Test Functions Port Status
boxo/gateway/*_test.go Gateway handler tests NOT PORTED
boxo/gateway/gateway_test.go Core gateway tests NOT PORTED
boxo/gateway/handler_test.go Request handler tests NOT PORTED
boxo/gateway/metrics_test.go Gateway metrics tests NOT PORTED

Phase 14: HTTP RPC API

Kubo Tests:

File Test Functions Port Status
kubo/core/corehttp/*_test.go HTTP API tests NOT PORTED
kubo/core/corehttp/corehttp_test.go Core HTTP tests NOT PORTED
kubo/core/corehttp/commands_test.go Command API tests NOT PORTED
kubo/client/rpc/*_test.go RPC client tests NOT PORTED

Phase 15: Remaining Commands and Polish

Kubo Tests:

File Test Functions Port Status
kubo/core/commands/diag_test.go Diagnostic command tests NOT PORTED
kubo/core/commands/log_test.go Log command tests NOT PORTED
kubo/core/commands/mount_test.go Mount command tests NOT PORTED
kubo/core/commands/p2p_test.go P2P command tests NOT PORTED
kubo/core/commands/stat_test.go Stat command tests NOT PORTED
kubo/core/commands/swarm_test.go Swarm command tests NOT PORTED
kubo/core/commands/version_test.go Version command tests NOT PORTED
kubo/test/cli/*_test.go CLI integration tests NOT PORTED

Sharness Tests

Kubo includes a comprehensive sharness-based CLI test suite in kubo/test/sharness/. These shell-based tests should also be ported:

  1. Location: kubo/test/sharness/t*.sh
  2. Porting Strategy: Convert to Rust integration tests using Command to invoke the CLI
  3. Priority: Port sharness tests that cover functionality not already tested by unit tests

Compliance Report Format

Integration tests produce a JSON report and human-readable summary:

================================================================================
FERRIPFS PHASE [N] COMPLIANCE REPORT
Generated: [timestamp]
Kubo Reference: v0.39.0
================================================================================

REQUIREMENT: [REQ-N.M] [Requirement Title]
------------------------------------------------------------------------------
Description: [What this requirement specifies]
Kubo Source: [Path to reference Go file]
Test: [test_name]
Status: PASS | FAIL

How to Verify:
  [Step-by-step instructions for human verification]

Evidence:
  [Captured output, file contents, or observable behavior]

------------------------------------------------------------------------------

PHASE SUMMARY
------------------------------------------------------------------------------
Total Requirements: [X]
Passed: [Y]
Failed: [Z]
Coverage: [XX.X%]
Status: COMPLIANT | NON-COMPLIANT
================================================================================

End-of-Phase Demonstration Requirements

At the completion of each phase, the implementer MUST provide a detailed compliance demonstration that includes:

  1. Individual Requirement Listing: Each requirement must be listed separately with its ID and title
  2. Test Validation: For each requirement, identify the specific test function that validates it
  3. Human Verification Guide: For each requirement, provide clear instructions explaining how a human reviewer can independently verify compliance based on the test output

The demonstration format for each requirement:

### REQ-X.Y: [Requirement Title]

**Test**: `test_function_name` in `path/to/test_file.rs`

**Status**: PASS | FAIL

**Evidence from test output**:
[Relevant excerpt from the test output showing the validation]

**How to verify manually**:
1. [Step-by-step instructions]
2. [What to look for in the output]
3. [Expected values or patterns that confirm compliance]

This ensures:

  • Complete traceability from requirement to test to evidence
  • Any reviewer can independently confirm each requirement was met
  • The demonstration serves as living documentation of compliance

Test Infrastructure Crate

Create ferripfs-test-harness crate providing:

/// A requirement that must be tested
pub struct Requirement {
    pub id: String,           // e.g., "REQ-1.3"
    pub title: String,
    pub description: String,
    pub kubo_source: String,  // e.g., "kubo/config/config.go:45-120"
    pub verification_steps: Vec<String>,
}

/// Result of testing a requirement
pub struct RequirementResult {
    pub requirement: Requirement,
    pub test_name: String,
    pub status: TestStatus,
    pub evidence: String,
    pub duration: Duration,
}

/// Generate human-readable compliance report
pub fn generate_compliance_report(phase: u8, results: Vec<RequirementResult>) -> String;

Phase 1: Project Foundation

Phase 1 Requirements

ID Requirement Kubo Source Test Type
REQ-1.1 Cargo workspace compiles with no errors N/A Build
REQ-1.2 LICENSE-MIT matches Kubo LICENSE-MIT exactly LICENSE-MIT Unit
REQ-1.3 LICENSE-APACHE matches Kubo LICENSE-APACHE exactly LICENSE-APACHE Unit
REQ-1.4 README.md states this is a port of Kubo v0.39.0 N/A Unit
REQ-1.5 ATTRIBUTION.md acknowledges Kubo, Protocol Labs, IPFS contributors N/A Unit
REQ-1.6 PORTING_GUIDE.md specifies file header format N/A Unit
REQ-1.7 PARITY.md lists all 131 Kubo commands with status tracking core/commands/root.go Unit
REQ-1.8 Test harness can generate compliance reports N/A Integration
REQ-1.9 ferripfs --version outputs version string cmd/ipfs/kubo/version.go Integration
REQ-1.10 ferripfs --help outputs help text with command list core/commands/root.go Integration

Phase 1 Directory Structure

ferripfs/
├── Cargo.toml
├── LICENSE-MIT
├── LICENSE-APACHE
├── README.md
├── ATTRIBUTION.md
├── PORTING_GUIDE.md
├── PARITY.md
├── crates/
│   └── ferripfs-test-harness/
│       ├── Cargo.toml
│       └── src/
│           ├── lib.rs
│           ├── requirement.rs
│           ├── report.rs
│           └── runner.rs
├── bin/
│   └── ferripfs/
│       ├── Cargo.toml
│       └── src/
│           └── main.rs
└── tests/
    └── phase1/
        ├── compliance.rs
        └── fixtures/

Phase 1 Integration Tests

REQ-1.2: License MIT Match

#[test]
fn req_1_2_license_mit_matches_kubo() {
    let req = Requirement {
        id: "REQ-1.2".into(),
        title: "LICENSE-MIT matches Kubo".into(),
        description: "The LICENSE-MIT file must be byte-for-byte identical to Kubo's LICENSE-MIT".into(),
        kubo_source: "LICENSE-MIT".into(),
        verification_steps: vec![
            "Read ferripfs/LICENSE-MIT".into(),
            "Read kubo/LICENSE-MIT".into(),
            "Compare byte-for-byte".into(),
            "Files must be identical".into(),
        ],
    };

    let ferripfs_license = fs::read_to_string("LICENSE-MIT").unwrap();
    let kubo_license = fs::read_to_string("../kubo/LICENSE-MIT").unwrap();

    let status = if ferripfs_license == kubo_license {
        TestStatus::Pass
    } else {
        TestStatus::Fail
    };

    let evidence = format!(
        "ferripfs LICENSE-MIT length: {} bytes\n\
         kubo LICENSE-MIT length: {} bytes\n\
         Match: {}",
        ferripfs_license.len(),
        kubo_license.len(),
        ferripfs_license == kubo_license
    );

    record_result(RequirementResult {
        requirement: req,
        test_name: "req_1_2_license_mit_matches_kubo".into(),
        status,
        evidence,
        duration: elapsed,
    });
}

REQ-1.7: Parity Document Completeness

#[test]
fn req_1_7_parity_lists_all_commands() {
    let req = Requirement {
        id: "REQ-1.7".into(),
        title: "PARITY.md lists all Kubo commands".into(),
        description: "PARITY.md must list all 131 Kubo CLI commands with tracking status".into(),
        kubo_source: "core/commands/root.go".into(),
        verification_steps: vec![
            "Parse Kubo core/commands/root.go for command list".into(),
            "Parse PARITY.md for listed commands".into(),
            "Every Kubo command must appear in PARITY.md".into(),
            "Each entry must have status indicator (not started/in progress/complete)".into(),
        ],
    };

    let kubo_commands = extract_kubo_commands("../kubo/core/commands/");
    let parity_commands = parse_parity_md("PARITY.md");

    let missing: Vec<_> = kubo_commands.iter()
        .filter(|c| !parity_commands.contains(c))
        .collect();

    let status = if missing.is_empty() { TestStatus::Pass } else { TestStatus::Fail };

    let evidence = format!(
        "Kubo commands found: {}\n\
         PARITY.md entries: {}\n\
         Missing commands: {:?}",
        kubo_commands.len(),
        parity_commands.len(),
        missing
    );

    // ... record result
}

REQ-1.9: Version Command

#[test]
fn req_1_9_version_command() {
    let req = Requirement {
        id: "REQ-1.9".into(),
        title: "Version command outputs version string".into(),
        description: "ferripfs --version must output version in format 'ferripfs version X.Y.Z (kubo v0.39.0 port)'".into(),
        kubo_source: "cmd/ipfs/kubo/version.go".into(),
        verification_steps: vec![
            "Run: ferripfs --version".into(),
            "Output must contain 'ferripfs version'".into(),
            "Output must contain 'kubo v0.39.0 port'".into(),
            "Exit code must be 0".into(),
        ],
    };

    let output = Command::new("./target/debug/ferripfs")
        .arg("--version")
        .output()
        .unwrap();

    let stdout = String::from_utf8_lossy(&output.stdout);
    let has_version = stdout.contains("ferripfs version");
    let has_kubo_ref = stdout.contains("kubo v0.39.0 port");
    let exit_ok = output.status.success();

    let status = if has_version && has_kubo_ref && exit_ok {
        TestStatus::Pass
    } else {
        TestStatus::Fail
    };

    let evidence = format!(
        "Command: ferripfs --version\n\
         Exit code: {}\n\
         stdout:\n{}\n\
         Contains 'ferripfs version': {}\n\
         Contains 'kubo v0.39.0 port': {}",
        output.status.code().unwrap_or(-1),
        stdout,
        has_version,
        has_kubo_ref
    );

    // ... record result
}

Phase 1 Compliance Verification

Run: cargo test --test phase1_compliance -- --nocapture

Output:

================================================================================
FERRIPFS PHASE 1 COMPLIANCE REPORT
Generated: 2024-XX-XX HH:MM:SS UTC
Kubo Reference: v0.39.0
================================================================================

REQUIREMENT: [REQ-1.1] Cargo workspace compiles
------------------------------------------------------------------------------
Description: Cargo workspace must compile with no errors or warnings
Kubo Source: N/A
Test: req_1_1_workspace_compiles
Status: PASS

How to Verify:
  1. Run: cargo build --workspace
  2. Observe zero errors
  3. Observe zero warnings (with -D warnings)

Evidence:
  cargo build exit code: 0
  Errors: 0
  Warnings: 0

------------------------------------------------------------------------------

REQUIREMENT: [REQ-1.2] LICENSE-MIT matches Kubo
------------------------------------------------------------------------------
Description: The LICENSE-MIT file must be byte-for-byte identical to Kubo's LICENSE-MIT
Kubo Source: LICENSE-MIT
Test: req_1_2_license_mit_matches_kubo
Status: PASS

How to Verify:
  1. Read ferripfs/LICENSE-MIT
  2. Read kubo/LICENSE-MIT
  3. Compare byte-for-byte
  4. Files must be identical

Evidence:
  ferripfs LICENSE-MIT length: 1076 bytes
  kubo LICENSE-MIT length: 1076 bytes
  Match: true

------------------------------------------------------------------------------
[... continues for all requirements ...]

PHASE 1 SUMMARY
------------------------------------------------------------------------------
Total Requirements: 10
Passed: 10
Failed: 0
Coverage: 87.3%
Status: COMPLIANT
================================================================================

Phase 2: Configuration and Repository

Phase 2 Requirements

ID Requirement Kubo Source Test Type
REQ-2.1 Config struct deserializes Kubo config.json config/config.go Unit
REQ-2.2 Config struct serializes to Kubo-compatible JSON config/config.go Unit
REQ-2.3 Identity config section: PeerID, PrivKey config/identity.go Unit
REQ-2.4 Datastore config section: Spec, Type config/datastore.go Unit
REQ-2.5 Addresses config: Swarm, API, Gateway, Announce, NoAnnounce config/addresses.go Unit
REQ-2.6 Bootstrap config: list of multiaddrs config/bootstrap_peers.go Unit
REQ-2.7 Swarm config: ConnMgr, Transports, RelayClient config/swarm.go Unit
REQ-2.8 Routing config: Type, Routers config/routing.go Unit
REQ-2.9 FSRepo initializes directory structure repo/fsrepo/fsrepo.go Integration
REQ-2.10 FSRepo creates config file repo/fsrepo/fsrepo.go Integration
REQ-2.11 FSRepo creates datastore directory repo/fsrepo/fsrepo.go Integration
REQ-2.12 FSRepo creates blocks directory repo/fsrepo/fsrepo.go Integration
REQ-2.13 FSRepo version file contains "18" repo/fsrepo/fsrepo.go:28 Integration
REQ-2.14 FSRepo lock prevents concurrent access repo/fsrepo/lock.go Integration
REQ-2.15 Keystore stores Ed25519 private keys repo/fsrepo/keystore/keystore.go Unit
REQ-2.16 Keystore stores RSA private keys repo/fsrepo/keystore/keystore.go Unit
REQ-2.17 ferripfs init creates valid repo cmd/ipfs/kubo/init.go Integration
REQ-2.18 ferripfs init generates peer identity cmd/ipfs/kubo/init.go Integration
REQ-2.19 ferripfs init --profile=server applies profile config/profile.go Integration
REQ-2.20 ferripfs config show outputs JSON config core/commands/config.go Integration
REQ-2.21 ferripfs config <key> gets config value core/commands/config.go Integration
REQ-2.22 ferripfs config <key> <value> sets config value core/commands/config.go Integration
REQ-2.23 ferripfs config replace replaces config file core/commands/config.go Integration
REQ-2.24 Config migration from Kubo repo works N/A Integration
REQ-2.25 Datastore interface: Put, Get, Has, Delete, Query repo/repo.go Unit
REQ-2.26 FlatFS datastore implementation go-ds-flatfs Integration
REQ-2.27 Datastore batching support go-datastore Unit

Phase 2 Crate Structure

crates/
├── ferripfs-config/
│   ├── Cargo.toml
│   └── src/
│       ├── lib.rs
│       ├── config.rs        # Main Config struct
│       ├── identity.rs      # Identity section
│       ├── datastore.rs     # Datastore section
│       ├── addresses.rs     # Addresses section
│       ├── bootstrap.rs     # Bootstrap peers
│       ├── swarm.rs         # Swarm settings
│       ├── routing.rs       # Routing config
│       ├── gateway.rs       # Gateway settings
│       ├── api.rs           # API settings
│       ├── profile.rs       # Config profiles
│       └── tests/
│           ├── kubo_compat.rs  # Test against real Kubo configs
│           └── fixtures/
│               └── kubo_config.json
├── ferripfs-repo/
│   ├── Cargo.toml
│   └── src/
│       ├── lib.rs
│       ├── repo.rs          # Repo trait
│       ├── fsrepo.rs        # Filesystem repo
│       ├── lock.rs          # Repo locking
│       ├── keystore.rs      # Key storage
│       ├── datastore/
│       │   ├── mod.rs
│       │   ├── traits.rs    # Datastore trait
│       │   ├── flatfs.rs    # FlatFS implementation
│       │   ├── memory.rs    # In-memory (for tests)
│       │   └── batch.rs     # Batching support
│       └── version.rs       # Repo versioning

Phase 2 Integration Tests

REQ-2.1: Config Deserialization

#[test]
fn req_2_1_deserialize_kubo_config() {
    let req = Requirement {
        id: "REQ-2.1".into(),
        title: "Config struct deserializes Kubo config.json".into(),
        description: "ferripfs Config must successfully parse a config.json generated by Kubo".into(),
        kubo_source: "config/config.go".into(),
        verification_steps: vec![
            "Generate config.json using: kubo init --repo=/tmp/kubo-test".into(),
            "Read /tmp/kubo-test/config".into(),
            "Deserialize using ferripfs_config::Config::from_reader()".into(),
            "All fields must parse without error".into(),
            "Identity.PeerID must be valid PeerId".into(),
            "Addresses.Swarm must contain at least one multiaddr".into(),
        ],
    };

    // Generate Kubo config
    let kubo_repo = tempdir().unwrap();
    Command::new("ipfs")
        .args(["init", "--repo", kubo_repo.path().to_str().unwrap()])
        .output()
        .expect("Kubo must be installed for interop tests");

    let config_path = kubo_repo.path().join("config");
    let config_json = fs::read_to_string(&config_path).unwrap();

    let result = Config::from_str(&config_json);

    let status = match &result {
        Ok(config) => {
            let peer_id_valid = config.identity.peer_id.parse::<PeerId>().is_ok();
            let has_swarm_addr = !config.addresses.swarm.is_empty();
            if peer_id_valid && has_swarm_addr {
                TestStatus::Pass
            } else {
                TestStatus::Fail
            }
        }
        Err(_) => TestStatus::Fail,
    };

    let evidence = match &result {
        Ok(config) => format!(
            "Kubo config parsed successfully\n\
             Config file size: {} bytes\n\
             PeerID: {}\n\
             Swarm addresses: {:?}\n\
             API address: {:?}\n\
             Gateway address: {:?}\n\
             Bootstrap peers: {} entries",
            config_json.len(),
            config.identity.peer_id,
            config.addresses.swarm,
            config.addresses.api,
            config.addresses.gateway,
            config.bootstrap.len()
        ),
        Err(e) => format!("Parse error: {}", e),
    };

    // ... record result
}

REQ-2.17: Init Command

#[test]
fn req_2_17_init_creates_valid_repo() {
    let req = Requirement {
        id: "REQ-2.17".into(),
        title: "ferripfs init creates valid repository".into(),
        description: "Running 'ferripfs init' must create a complete, valid IPFS repository".into(),
        kubo_source: "cmd/ipfs/kubo/init.go".into(),
        verification_steps: vec![
            "Create empty temp directory".into(),
            "Run: IPFS_PATH=/tmp/test ferripfs init".into(),
            "Verify exit code is 0".into(),
            "Verify /tmp/test/config exists and is valid JSON".into(),
            "Verify /tmp/test/datastore directory exists".into(),
            "Verify /tmp/test/blocks directory exists".into(),
            "Verify /tmp/test/version contains '18'".into(),
            "Verify config contains valid Identity.PeerID".into(),
            "Verify generated PeerID matches Identity.PrivKey".into(),
        ],
    };

    let repo_path = tempdir().unwrap();
    let repo_str = repo_path.path().to_str().unwrap();

    let output = Command::new("./target/debug/ferripfs")
        .env("IPFS_PATH", repo_str)
        .arg("init")
        .output()
        .unwrap();

    let config_exists = repo_path.path().join("config").exists();
    let datastore_exists = repo_path.path().join("datastore").is_dir();
    let blocks_exists = repo_path.path().join("blocks").is_dir();
    let version_content = fs::read_to_string(repo_path.path().join("version"))
        .unwrap_or_default();

    let config_valid = if config_exists {
        let config_str = fs::read_to_string(repo_path.path().join("config")).unwrap();
        Config::from_str(&config_str).is_ok()
    } else {
        false
    };

    let all_pass = output.status.success()
        && config_exists
        && config_valid
        && datastore_exists
        && blocks_exists
        && version_content.trim() == "18";

    let status = if all_pass { TestStatus::Pass } else { TestStatus::Fail };

    let evidence = format!(
        "Command: IPFS_PATH={} ferripfs init\n\
         Exit code: {}\n\
         stdout: {}\n\
         stderr: {}\n\
         \n\
         Repository structure:\n\
         - config exists: {} (valid JSON: {})\n\
         - datastore/ exists: {}\n\
         - blocks/ exists: {}\n\
         - version content: '{}'",
        repo_str,
        output.status.code().unwrap_or(-1),
        String::from_utf8_lossy(&output.stdout),
        String::from_utf8_lossy(&output.stderr),
        config_exists,
        config_valid,
        datastore_exists,
        blocks_exists,
        version_content.trim()
    );

    // ... record result
}

REQ-2.24: Kubo Migration

#[test]
fn req_2_24_kubo_repo_migration() {
    let req = Requirement {
        id: "REQ-2.24".into(),
        title: "Config migration from existing Kubo repo".into(),
        description: "ferripfs must be able to use an existing Kubo-initialized repository".into(),
        kubo_source: "N/A".into(),
        verification_steps: vec![
            "Initialize repo with Kubo: ipfs init".into(),
            "Point ferripfs at same repo: IPFS_PATH=<kubo-repo>".into(),
            "Run: ferripfs config show".into(),
            "Config must parse and display correctly".into(),
            "Run: ferripfs id".into(),
            "PeerID must match Kubo's PeerID".into(),
        ],
    };

    // Initialize with Kubo
    let repo_path = tempdir().unwrap();
    let repo_str = repo_path.path().to_str().unwrap();

    Command::new("ipfs")
        .env("IPFS_PATH", repo_str)
        .arg("init")
        .output()
        .expect("Kubo required");

    // Get Kubo's peer ID
    let kubo_id = Command::new("ipfs")
        .env("IPFS_PATH", repo_str)
        .args(["config", "Identity.PeerID"])
        .output()
        .unwrap();
    let kubo_peer_id = String::from_utf8_lossy(&kubo_id.stdout).trim().to_string();

    // Use ferripfs with same repo
    let ferripfs_config = Command::new("./target/debug/ferripfs")
        .env("IPFS_PATH", repo_str)
        .args(["config", "show"])
        .output()
        .unwrap();

    let ferripfs_id = Command::new("./target/debug/ferripfs")
        .env("IPFS_PATH", repo_str)
        .arg("id")
        .output()
        .unwrap();

    let config_parsed = serde_json::from_slice::<serde_json::Value>(&ferripfs_config.stdout).is_ok();
    let id_output = String::from_utf8_lossy(&ferripfs_id.stdout);
    let id_matches = id_output.contains(&kubo_peer_id);

    let status = if config_parsed && id_matches && ferripfs_config.status.success() {
        TestStatus::Pass
    } else {
        TestStatus::Fail
    };

    let evidence = format!(
        "Kubo repository: {}\n\
         Kubo PeerID: {}\n\
         \n\
         ferripfs config show:\n\
         Exit code: {}\n\
         Valid JSON: {}\n\
         \n\
         ferripfs id:\n\
         Exit code: {}\n\
         Contains Kubo PeerID: {}\n\
         Output: {}",
        repo_str,
        kubo_peer_id,
        ferripfs_config.status.code().unwrap_or(-1),
        config_parsed,
        ferripfs_id.status.code().unwrap_or(-1),
        id_matches,
        id_output
    );

    // ... record result
}

Phase 3: Blockstore and Content Addressing

Phase 3 Requirements

ID Requirement Kubo Source Test Type
REQ-3.1 Blockstore trait: Get, Put, Has, Delete, AllKeysChan boxo/blockstore/blockstore.go Unit
REQ-3.2 Block identified by CID go-block-format Unit
REQ-3.3 CIDv0 (sha2-256, dag-pb) support go-cid Unit
REQ-3.4 CIDv1 (multiple codecs, hashes) support go-cid Unit
REQ-3.5 Blockstore persists to FlatFS datastore boxo/blockstore Integration
REQ-3.6 Blockstore retrieves persisted blocks boxo/blockstore Integration
REQ-3.7 GCBlockstore prevents deletion during pin boxo/blockstore/gcblocker.go Unit
REQ-3.8 Block hashing matches Kubo for same content N/A Integration
REQ-3.9 ferripfs block put stores raw block core/commands/block.go Integration
REQ-3.10 ferripfs block get <cid> retrieves block core/commands/block.go Integration
REQ-3.11 ferripfs block stat <cid> shows size core/commands/block.go Integration
REQ-3.12 ferripfs block rm <cid> removes block core/commands/block.go Integration
REQ-3.13 Block put with --format=raw uses raw codec core/commands/block.go Integration
REQ-3.14 Block put with --mhtype=sha2-512 uses sha2-512 core/commands/block.go Integration
REQ-3.15 Caching blockstore reduces disk reads boxo/blockstore/caching.go Unit

Phase 3 Integration Tests

REQ-3.8: Hash Compatibility with Kubo

#[test]
fn req_3_8_block_hash_matches_kubo() {
    let req = Requirement {
        id: "REQ-3.8".into(),
        title: "Block hashing matches Kubo".into(),
        description: "Given identical content, ferripfs must produce the same CID as Kubo".into(),
        kubo_source: "N/A (interoperability requirement)".into(),
        verification_steps: vec![
            "Create test file with known content".into(),
            "Add to Kubo: echo 'hello world' | ipfs block put".into(),
            "Add to ferripfs: echo 'hello world' | ferripfs block put".into(),
            "CIDs must be identical".into(),
            "Repeat with various content sizes: 0 bytes, 1 byte, 256KB, 1MB".into(),
        ],
    };

    let test_cases = vec![
        ("empty", b"".to_vec()),
        ("hello", b"hello world".to_vec()),
        ("newline", b"hello world\n".to_vec()),
        ("256kb", vec![0x42u8; 256 * 1024]),
        ("1mb", vec![0xFFu8; 1024 * 1024]),
    ];

    let mut results = Vec::new();

    for (name, content) in &test_cases {
        // Get CID from Kubo
        let kubo = Command::new("ipfs")
            .args(["block", "put", "--format=raw"])
            .stdin(Stdio::piped())
            .stdout(Stdio::piped())
            .spawn()
            .unwrap();
        kubo.stdin.unwrap().write_all(content).unwrap();
        let kubo_cid = String::from_utf8_lossy(&kubo.wait_with_output().unwrap().stdout)
            .trim().to_string();

        // Get CID from ferripfs
        let ferripfs = Command::new("./target/debug/ferripfs")
            .args(["block", "put", "--format=raw"])
            .stdin(Stdio::piped())
            .stdout(Stdio::piped())
            .spawn()
            .unwrap();
        ferripfs.stdin.unwrap().write_all(content).unwrap();
        let ferripfs_cid = String::from_utf8_lossy(&ferripfs.wait_with_output().unwrap().stdout)
            .trim().to_string();

        results.push((name, content.len(), kubo_cid.clone(), ferripfs_cid.clone(), kubo_cid == ferripfs_cid));
    }

    let all_match = results.iter().all(|(_, _, _, _, matches)| *matches);
    let status = if all_match { TestStatus::Pass } else { TestStatus::Fail };

    let evidence = results.iter()
        .map(|(name, size, kubo, ferripfs, matches)| {
            format!(
                "Test '{}' ({} bytes):\n  Kubo CID:     {}\n  ferripfs CID: {}\n  Match: {}",
                name, size, kubo, ferripfs, matches
            )
        })
        .collect::<Vec<_>>()
        .join("\n\n");

    // ... record result
}

Phase 4: UnixFS and File Operations

Phase 4 Requirements

ID Requirement Kubo Source Test Type
REQ-4.1 Chunker: size-based (default 256KB) boxo/chunker/splitting.go Unit
REQ-4.2 Chunker: rabin fingerprinting boxo/chunker/rabin.go Unit
REQ-4.3 DAG builder: balanced layout boxo/ipld/unixfs/importer/balanced Unit
REQ-4.4 DAG builder: trickle layout boxo/ipld/unixfs/importer/trickle Unit
REQ-4.5 UnixFS protobuf encoding matches Kubo boxo/ipld/unixfs/pb Unit
REQ-4.6 File node type encoding boxo/ipld/unixfs/unixfs.go Unit
REQ-4.7 Directory node type encoding boxo/ipld/unixfs/unixfs.go Unit
REQ-4.8 Symlink node type encoding boxo/ipld/unixfs/unixfs.go Unit
REQ-4.9 HAMT sharded directories (>1000 entries) boxo/ipld/unixfs/hamt Integration
REQ-4.10 Raw leaves mode (--raw-leaves) core/commands/add.go Integration
REQ-4.11 CID version selection (--cid-version) core/commands/add.go Integration
REQ-4.12 ferripfs add <file> returns CID core/commands/add.go Integration
REQ-4.13 ferripfs add -r <dir> adds directory recursively core/commands/add.go Integration
REQ-4.14 ferripfs add --wrap-with-directory wraps file core/commands/add.go Integration
REQ-4.15 ferripfs add --pin=false does not pin core/commands/add.go Integration
REQ-4.16 ferripfs add --progress shows progress core/commands/add.go Integration
REQ-4.17 ferripfs add CID matches Kubo for same file N/A Integration
REQ-4.18 ferripfs cat <cid> outputs file content core/commands/cat.go Integration
REQ-4.19 ferripfs cat --offset --length partial read core/commands/cat.go Integration
REQ-4.20 ferripfs get <cid> downloads to file core/commands/get.go Integration
REQ-4.21 ferripfs get -o <path> specifies output core/commands/get.go Integration
REQ-4.22 ferripfs ls <cid> lists directory entries core/commands/ls.go Integration
REQ-4.23 ferripfs ls --resolve-type resolves types core/commands/ls.go Integration
REQ-4.24 ferripfs refs <cid> lists block references core/commands/refs.go Integration
REQ-4.25 ferripfs refs -r lists recursively core/commands/refs.go Integration
REQ-4.26 Content added by Kubo can be read by ferripfs N/A Integration
REQ-4.27 Content added by ferripfs can be read by Kubo N/A Integration

Phase 4 Critical Integration Tests

REQ-4.17: Add CID Compatibility

#[test]
fn req_4_17_add_cid_matches_kubo() {
    let req = Requirement {
        id: "REQ-4.17".into(),
        title: "ferripfs add produces same CID as Kubo".into(),
        description: "Adding identical files must produce identical CIDs".into(),
        kubo_source: "core/commands/add.go".into(),
        verification_steps: vec![
            "Create test files of various sizes and types".into(),
            "Add each file to Kubo: ipfs add <file>".into(),
            "Add each file to ferripfs: ferripfs add <file>".into(),
            "Compare CIDs - must be identical".into(),
            "Test with: empty file, small text, binary, large file, directory".into(),
        ],
    };

    let test_cases = vec![
        ("empty.txt", vec![]),
        ("hello.txt", b"hello world\n".to_vec()),
        ("binary.bin", (0u8..=255).collect()),
        ("large.bin", vec![0x42u8; 1024 * 1024]),  // 1MB
    ];

    let kubo_repo = tempdir().unwrap();
    let ferripfs_repo = tempdir().unwrap();

    // Initialize repos
    init_kubo_repo(&kubo_repo);
    init_ferripfs_repo(&ferripfs_repo);

    let mut results = Vec::new();

    for (name, content) in &test_cases {
        let file_path = kubo_repo.path().join(name);
        fs::write(&file_path, content).unwrap();

        let kubo_output = Command::new("ipfs")
            .env("IPFS_PATH", kubo_repo.path())
            .args(["add", "-q", file_path.to_str().unwrap()])
            .output()
            .unwrap();
        let kubo_cid = String::from_utf8_lossy(&kubo_output.stdout).trim().to_string();

        let ferripfs_output = Command::new("./target/debug/ferripfs")
            .env("IPFS_PATH", ferripfs_repo.path())
            .args(["add", "-q", file_path.to_str().unwrap()])
            .output()
            .unwrap();
        let ferripfs_cid = String::from_utf8_lossy(&ferripfs_output.stdout).trim().to_string();

        results.push((name.to_string(), content.len(), kubo_cid.clone(), ferripfs_cid.clone(), kubo_cid == ferripfs_cid));
    }

    // Directory test
    let dir_path = kubo_repo.path().join("testdir");
    fs::create_dir(&dir_path).unwrap();
    fs::write(dir_path.join("a.txt"), b"file a").unwrap();
    fs::write(dir_path.join("b.txt"), b"file b").unwrap();

    let kubo_dir = Command::new("ipfs")
        .env("IPFS_PATH", kubo_repo.path())
        .args(["add", "-rq", dir_path.to_str().unwrap()])
        .output()
        .unwrap();
    let kubo_dir_cid = String::from_utf8_lossy(&kubo_dir.stdout)
        .lines().last().unwrap_or("").to_string();

    let ferripfs_dir = Command::new("./target/debug/ferripfs")
        .env("IPFS_PATH", ferripfs_repo.path())
        .args(["add", "-rq", dir_path.to_str().unwrap()])
        .output()
        .unwrap();
    let ferripfs_dir_cid = String::from_utf8_lossy(&ferripfs_dir.stdout)
        .lines().last().unwrap_or("").to_string();

    results.push(("directory".into(), 0, kubo_dir_cid.clone(), ferripfs_dir_cid.clone(), kubo_dir_cid == ferripfs_dir_cid));

    let all_match = results.iter().all(|(_, _, _, _, m)| *m);
    let status = if all_match { TestStatus::Pass } else { TestStatus::Fail };

    let evidence = format!(
        "CID Comparison Results:\n\n{}\n\nAll CIDs match: {}",
        results.iter()
            .map(|(name, size, kubo, ferripfs, matches)| {
                format!(
                    "{}:\n  Size: {} bytes\n  Kubo:     {}\n  ferripfs: {}\n  Match: {}",
                    name, size, kubo, ferripfs, if *matches { "YES" } else { "NO" }
                )
            })
            .collect::<Vec<_>>()
            .join("\n\n"),
        all_match
    );

    // ... record result
}

REQ-4.26 & REQ-4.27: Cross-Node Interoperability

#[test]
fn req_4_26_27_cross_node_content_exchange() {
    let req = Requirement {
        id: "REQ-4.26/27".into(),
        title: "Cross-node content interoperability".into(),
        description: "Content added by one implementation must be readable by the other".into(),
        kubo_source: "N/A (interoperability requirement)".into(),
        verification_steps: vec![
            "Add file with Kubo, read with ferripfs (shared repo)".into(),
            "Add file with ferripfs, read with Kubo (shared repo)".into(),
            "Compare retrieved content with original".into(),
            "Test with various file sizes".into(),
        ],
    };

    let shared_repo = tempdir().unwrap();
    init_kubo_repo(&shared_repo);

    // Test 1: Kubo adds, ferripfs reads
    let content1 = b"Content added by Kubo\n";
    let file1 = shared_repo.path().join("kubo_file.txt");
    fs::write(&file1, content1).unwrap();

    let kubo_add = Command::new("ipfs")
        .env("IPFS_PATH", shared_repo.path())
        .args(["add", "-q", file1.to_str().unwrap()])
        .output()
        .unwrap();
    let cid1 = String::from_utf8_lossy(&kubo_add.stdout).trim().to_string();

    let ferripfs_cat = Command::new("./target/debug/ferripfs")
        .env("IPFS_PATH", shared_repo.path())
        .args(["cat", &cid1])
        .output()
        .unwrap();
    let retrieved1 = ferripfs_cat.stdout.clone();

    // Test 2: ferripfs adds, Kubo reads
    let content2 = b"Content added by ferripfs\n";
    let file2 = shared_repo.path().join("ferripfs_file.txt");
    fs::write(&file2, content2).unwrap();

    let ferripfs_add = Command::new("./target/debug/ferripfs")
        .env("IPFS_PATH", shared_repo.path())
        .args(["add", "-q", file2.to_str().unwrap()])
        .output()
        .unwrap();
    let cid2 = String::from_utf8_lossy(&ferripfs_add.stdout).trim().to_string();

    let kubo_cat = Command::new("ipfs")
        .env("IPFS_PATH", shared_repo.path())
        .args(["cat", &cid2])
        .output()
        .unwrap();
    let retrieved2 = kubo_cat.stdout.clone();

    let test1_pass = retrieved1 == content1;
    let test2_pass = retrieved2 == content2;

    let status = if test1_pass && test2_pass { TestStatus::Pass } else { TestStatus::Fail };

    let evidence = format!(
        "Test 1: Kubo adds, ferripfs reads\n\
         Original content: {:?}\n\
         CID: {}\n\
         Retrieved by ferripfs: {:?}\n\
         Match: {}\n\
         \n\
         Test 2: ferripfs adds, Kubo reads\n\
         Original content: {:?}\n\
         CID: {}\n\
         Retrieved by Kubo: {:?}\n\
         Match: {}",
        String::from_utf8_lossy(content1),
        cid1,
        String::from_utf8_lossy(&retrieved1),
        test1_pass,
        String::from_utf8_lossy(content2),
        cid2,
        String::from_utf8_lossy(&retrieved2),
        test2_pass
    );

    // ... record result
}

Phase 5: Pinning System

Phase 5 Requirements

ID Requirement Kubo Source Test Type
REQ-5.1 Pin types: direct, recursive, indirect boxo/pinning/pinner/pin.go Unit
REQ-5.2 Pinner trait: IsPinned, Pin, Unpin, PinWithMode boxo/pinning/pinner/pin.go Unit
REQ-5.3 Recursive pin follows all links boxo/pinning/pinner/dspinner Unit
REQ-5.4 Direct pin only pins the root boxo/pinning/pinner/dspinner Unit
REQ-5.5 Indirect pins created by recursive pins boxo/pinning/pinner/dspinner Unit
REQ-5.6 Pin names support boxo/pinning/pinner/dspinner Unit
REQ-5.7 Pin persistence across restart boxo/pinning/pinner/dspinner Integration
REQ-5.8 ferripfs pin add <cid> pins recursively core/commands/pin/pin.go Integration
REQ-5.9 ferripfs pin add --recursive=false pins directly core/commands/pin/pin.go Integration
REQ-5.10 ferripfs pin add --name=<name> adds pin name core/commands/pin/pin.go Integration
REQ-5.11 ferripfs pin rm <cid> removes pin core/commands/pin/pin.go Integration
REQ-5.12 ferripfs pin ls lists all pins core/commands/pin/pin.go Integration
REQ-5.13 ferripfs pin ls --type=recursive filters core/commands/pin/pin.go Integration
REQ-5.14 ferripfs pin ls --names shows pin names core/commands/pin/pin.go Integration
REQ-5.15 ferripfs pin verify verifies pin integrity core/commands/pin/pin.go Integration
REQ-5.16 ferripfs pin update <old> <new> updates pin core/commands/pin/pin.go Integration
REQ-5.17 GC does not remove pinned blocks core/commands/repo.go Integration
REQ-5.18 GC removes unpinned blocks core/commands/repo.go Integration
REQ-5.19 ferripfs repo gc runs garbage collection core/commands/repo.go Integration

Phase 6: DAG Operations

Phase 6 Requirements

ID Requirement Kubo Source Test Type
REQ-6.1 IPLD DAG node creation boxo/ipld/merkledag Unit
REQ-6.2 dag-pb codec support go-codec-dagpb Unit
REQ-6.3 dag-cbor codec support go-ipld-cbor Unit
REQ-6.4 dag-json codec support go-ipld-prime/codec/dagjson Unit
REQ-6.5 raw codec support N/A Unit
REQ-6.6 ferripfs dag get <cid> retrieves node core/commands/dag/dag.go Integration
REQ-6.7 ferripfs dag get --output-codec transcodes core/commands/dag/dag.go Integration
REQ-6.8 ferripfs dag put stores DAG node core/commands/dag/dag.go Integration
REQ-6.9 ferripfs dag put --input-codec specifies input core/commands/dag/dag.go Integration
REQ-6.10 ferripfs dag put --store-codec specifies storage core/commands/dag/dag.go Integration
REQ-6.11 ferripfs dag resolve <path> resolves IPLD path core/commands/dag/dag.go Integration
REQ-6.12 ferripfs dag stat <cid> shows DAG statistics core/commands/dag/dag.go Integration
REQ-6.13 ferripfs dag export <cid> exports CAR core/commands/dag/dag.go Integration
REQ-6.14 ferripfs dag import <car> imports CAR core/commands/dag/dag.go Integration
REQ-6.15 Path traversal through DAG links boxo/path Unit

Phase 7: Networking (libp2p)

Phase 7 Requirements

ID Requirement Kubo Source Test Type
REQ-7.1 libp2p host initialization core/node/libp2p/host.go Unit
REQ-7.2 TCP transport core/node/libp2p/transport.go Integration
REQ-7.3 QUIC transport core/node/libp2p/transport.go Integration
REQ-7.4 WebSocket transport core/node/libp2p/transport.go Integration
REQ-7.5 Noise encryption core/node/libp2p/sec.go Unit
REQ-7.6 Yamux multiplexing core/node/libp2p/smux.go Unit
REQ-7.7 mDNS local discovery core/node/libp2p/discovery.go Integration
REQ-7.8 Connection manager (limits) core/node/libp2p/rcmgr.go Unit
REQ-7.9 NAT traversal (UPnP) core/node/libp2p/nat.go Integration
REQ-7.10 AutoNAT service core/node/libp2p/nat.go Integration
REQ-7.11 Hole punching core/node/libp2p/nat.go Integration
REQ-7.12 Circuit relay client core/node/libp2p/relay.go Integration
REQ-7.13 Identify protocol core/node/libp2p/host.go Integration
REQ-7.14 ferripfs daemon starts and listens cmd/ipfs/kubo/daemon.go Integration
REQ-7.15 ferripfs daemon announces addresses cmd/ipfs/kubo/daemon.go Integration
REQ-7.16 ferripfs id shows peer info core/commands/id.go Integration
REQ-7.17 ferripfs swarm peers lists connected peers core/commands/swarm.go Integration
REQ-7.18 ferripfs swarm connect <addr> connects to peer core/commands/swarm.go Integration
REQ-7.19 ferripfs swarm disconnect <addr> disconnects core/commands/swarm.go Integration
REQ-7.20 ferripfs swarm addrs lists known addresses core/commands/swarm.go Integration
REQ-7.21 ferripfs ping <peer> measures latency core/commands/ping.go Integration
REQ-7.22 ferripfs bootstrap list shows bootstrap peers core/commands/bootstrap.go Integration
REQ-7.23 ferripfs bootstrap add <addr> adds bootstrap core/commands/bootstrap.go Integration
REQ-7.24 ferripfs bootstrap rm <addr> removes bootstrap core/commands/bootstrap.go Integration
REQ-7.25 ferripfs shutdown stops daemon gracefully core/commands/shutdown.go Integration
REQ-7.26 ferripfs can connect to Kubo peer N/A Integration
REQ-7.27 Kubo can connect to ferripfs peer N/A Integration

Phase 8: DHT and Routing

Phase 8 Requirements

ID Requirement Kubo Source Test Type
REQ-8.1 Kademlia DHT client mode core/node/libp2p/routing.go Integration
REQ-8.2 Kademlia DHT server mode core/node/libp2p/routing.go Integration
REQ-8.3 DHT bootstrap go-libp2p-kad-dht Integration
REQ-8.4 Content routing: Provide go-libp2p-kad-dht Integration
REQ-8.5 Content routing: FindProviders go-libp2p-kad-dht Integration
REQ-8.6 Peer routing: FindPeer go-libp2p-kad-dht Integration
REQ-8.7 Value store: PutValue (IPNS) go-libp2p-kad-dht Integration
REQ-8.8 Value store: GetValue (IPNS) go-libp2p-kad-dht Integration
REQ-8.9 ferripfs dht findprovs <cid> finds providers core/commands/dht.go Integration
REQ-8.10 ferripfs dht findpeer <peerid> finds peer core/commands/dht.go Integration
REQ-8.11 ferripfs dht provide <cid> announces content core/commands/dht.go Integration
REQ-8.12 ferripfs routing get <key> retrieves value core/commands/routing.go Integration
REQ-8.13 ferripfs routing put <key> <value> stores value core/commands/routing.go Integration
REQ-8.14 ferripfs routing findprovs <cid> finds providers core/commands/routing.go Integration
REQ-8.15 ferripfs routing findpeer <peerid> finds peer core/commands/routing.go Integration
REQ-8.16 ferripfs stats dht shows DHT stats core/commands/stat_dht.go Integration

Phase 9: Bitswap and Content Exchange

Phase 9 Requirements

ID Requirement Kubo Source Test Type
REQ-9.1 Bitswap protocol /ipfs/bitswap/1.2.0 boxo/bitswap Integration
REQ-9.2 Bitswap wantlist management boxo/bitswap/wantlist Unit
REQ-9.3 Bitswap block requests boxo/bitswap/client Integration
REQ-9.4 Bitswap block serving boxo/bitswap/server Integration
REQ-9.5 Bitswap ledger tracking boxo/bitswap/decision Unit
REQ-9.6 ferripfs bitswap stat shows stats core/commands/bitswap.go Integration
REQ-9.7 ferripfs bitswap wantlist shows wanted blocks core/commands/bitswap.go Integration
REQ-9.8 ferripfs bitswap ledger <peer> shows ledger core/commands/bitswap.go Integration
REQ-9.9 Content exchange: ferripfs fetches from Kubo N/A Integration
REQ-9.10 Content exchange: Kubo fetches from ferripfs N/A Integration
REQ-9.11 Large file transfer (100MB+) N/A Integration
REQ-9.12 Concurrent block requests boxo/bitswap Integration

Phase 9 Critical Integration Test

REQ-9.9 & REQ-9.10: Cross-Implementation Content Exchange

#[test]
fn req_9_9_10_cross_impl_content_exchange() {
    let req = Requirement {
        id: "REQ-9.9/10".into(),
        title: "Content exchange between ferripfs and Kubo over network".into(),
        description: "Nodes must be able to fetch content from each other over the network".into(),
        kubo_source: "N/A (interoperability requirement)".into(),
        verification_steps: vec![
            "Start Kubo daemon on port 4001".into(),
            "Start ferripfs daemon on port 4002".into(),
            "Add unique content to Kubo".into(),
            "Add unique content to ferripfs".into(),
            "Connect nodes via swarm connect".into(),
            "Fetch Kubo content from ferripfs using CID".into(),
            "Fetch ferripfs content from Kubo using CID".into(),
            "Verify fetched content matches original".into(),
        ],
    };

    // Start Kubo daemon
    let kubo_repo = tempdir().unwrap();
    init_kubo_repo(&kubo_repo);
    let kubo_daemon = Command::new("ipfs")
        .env("IPFS_PATH", kubo_repo.path())
        .args(["daemon", "--offline=false"])
        .spawn()
        .unwrap();
    wait_for_daemon(&kubo_repo, 4001);

    // Start ferripfs daemon
    let ferripfs_repo = tempdir().unwrap();
    init_ferripfs_repo(&ferripfs_repo);
    let ferripfs_daemon = Command::new("./target/debug/ferripfs")
        .env("IPFS_PATH", ferripfs_repo.path())
        .args(["daemon"])
        .spawn()
        .unwrap();
    wait_for_daemon(&ferripfs_repo, 4002);

    // Add content to each
    let kubo_content = b"Content from Kubo node for cross-exchange test";
    let kubo_cid = add_content_to_kubo(&kubo_repo, kubo_content);

    let ferripfs_content = b"Content from ferripfs node for cross-exchange test";
    let ferripfs_cid = add_content_to_ferripfs(&ferripfs_repo, ferripfs_content);

    // Get peer addresses and connect
    let kubo_addr = get_peer_addr(&kubo_repo, "kubo");
    let ferripfs_addr = get_peer_addr(&ferripfs_repo, "ferripfs");

    connect_peers(&ferripfs_repo, &kubo_addr);

    // Cross-fetch
    let fetched_from_kubo = Command::new("./target/debug/ferripfs")
        .env("IPFS_PATH", ferripfs_repo.path())
        .args(["cat", &kubo_cid])
        .output()
        .unwrap();

    let fetched_from_ferripfs = Command::new("ipfs")
        .env("IPFS_PATH", kubo_repo.path())
        .args(["cat", &ferripfs_cid])
        .output()
        .unwrap();

    // Cleanup
    kubo_daemon.kill();
    ferripfs_daemon.kill();

    let kubo_fetch_ok = fetched_from_kubo.stdout == kubo_content;
    let ferripfs_fetch_ok = fetched_from_ferripfs.stdout == ferripfs_content;

    let status = if kubo_fetch_ok && ferripfs_fetch_ok {
        TestStatus::Pass
    } else {
        TestStatus::Fail
    };

    let evidence = format!(
        "Network Content Exchange Test\n\
         ==============================\n\
         \n\
         Kubo node:\n\
         - Address: {}\n\
         - Added content: {:?}\n\
         - CID: {}\n\
         \n\
         ferripfs node:\n\
         - Address: {}\n\
         - Added content: {:?}\n\
         - CID: {}\n\
         \n\
         Cross-fetch results:\n\
         - ferripfs fetched from Kubo: {} bytes, match: {}\n\
         - Kubo fetched from ferripfs: {} bytes, match: {}\n\
         \n\
         Overall: {}",
        kubo_addr,
        String::from_utf8_lossy(kubo_content),
        kubo_cid,
        ferripfs_addr,
        String::from_utf8_lossy(ferripfs_content),
        ferripfs_cid,
        fetched_from_kubo.stdout.len(),
        kubo_fetch_ok,
        fetched_from_ferripfs.stdout.len(),
        ferripfs_fetch_ok,
        if kubo_fetch_ok && ferripfs_fetch_ok { "SUCCESS" } else { "FAILURE" }
    );

    // ... record result
}

Phase 10: IPNS and Key Management

Phase 10 Requirements

ID Requirement Kubo Source Test Type
REQ-10.1 IPNS record protobuf format boxo/ipns/pb Unit
REQ-10.2 IPNS record signing (Ed25519) boxo/ipns Unit
REQ-10.3 IPNS record validation boxo/ipns Unit
REQ-10.4 IPNS record versioning (sequence) boxo/ipns Unit
REQ-10.5 IPNS publish to DHT boxo/namesys/publisher.go Integration
REQ-10.6 IPNS resolve from DHT boxo/namesys/resolver.go Integration
REQ-10.7 IPNS republisher service boxo/namesys/republisher Integration
REQ-10.8 Key generation: Ed25519 (default) core/commands/keystore.go Integration
REQ-10.9 Key generation: RSA core/commands/keystore.go Integration
REQ-10.10 Key generation: ECDSA core/commands/keystore.go Integration
REQ-10.11 Key generation: Secp256k1 core/commands/keystore.go Integration
REQ-10.12 ferripfs key gen <name> generates key core/commands/keystore.go Integration
REQ-10.13 ferripfs key list lists keys core/commands/keystore.go Integration
REQ-10.14 ferripfs key rename <old> <new> renames core/commands/keystore.go Integration
REQ-10.15 ferripfs key rm <name> removes key core/commands/keystore.go Integration
REQ-10.16 ferripfs key export <name> exports key core/commands/keystore.go Integration
REQ-10.17 ferripfs key import <name> <file> imports key core/commands/keystore.go Integration
REQ-10.18 ferripfs name publish <cid> publishes IPNS core/commands/name/publish.go Integration
REQ-10.19 ferripfs name publish --key=<name> uses key core/commands/name/publish.go Integration
REQ-10.20 ferripfs name resolve <name> resolves IPNS core/commands/name/resolve.go Integration
REQ-10.21 ferripfs name inspect <record> inspects core/commands/name/inspect.go Integration
REQ-10.22 IPNS record published by ferripfs resolved by Kubo N/A Integration
REQ-10.23 IPNS record published by Kubo resolved by ferripfs N/A Integration

Phases 11-15: Advanced Features

Phase 11: MFS (Mutable File System)

  • 20 requirements covering files subcommands
  • REQ-11.1 through REQ-11.20

Phase 12: PubSub

  • 10 requirements covering GossipSub/FloodSub
  • REQ-12.1 through REQ-12.10

Phase 13: HTTP Gateway

  • 15 requirements covering path/subdomain gateway
  • REQ-13.1 through REQ-13.15

Phase 14: HTTP RPC API

  • 10 requirements covering JSON-RPC parity
  • REQ-14.1 through REQ-14.10

Phase 15: Remaining Commands and Polish

  • All remaining CLI commands
  • Full sharness test suite porting
  • Performance benchmarking
  • Documentation completion

Continuous Integration

Per-Phase CI Pipeline

# .github/workflows/phase-N.yml
name: Phase N Compliance

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Kubo (for interop tests)
        run: |
          wget https://dist.ipfs.tech/kubo/v0.39.0/kubo_v0.39.0_linux-amd64.tar.gz
          tar xzf kubo_v0.39.0_linux-amd64.tar.gz
          sudo mv kubo/ipfs /usr/local/bin/

      - name: Build
        run: cargo build --workspace

      - name: Run Phase N Tests
        run: cargo test --test phase_n_compliance -- --nocapture

      - name: Check Coverage
        run: |
          cargo tarpaulin --out Json --output-dir coverage
          python3 scripts/check_coverage.py coverage/tarpaulin-report.json 85

      - name: Generate Compliance Report
        run: cargo test --test phase_n_compliance -- --nocapture > compliance_report.txt

      - name: Upload Compliance Report
        uses: actions/upload-artifact@v4
        with:
          name: phase-n-compliance-report
          path: compliance_report.txt

Summary

This plan provides:

  1. Granular requirements - Each phase has specific, testable requirements with IDs

  2. 85% coverage target - Enforced per phase via cargo-tarpaulin

  3. Human-readable compliance reports - Every integration test produces detailed output showing:

    • Requirement ID and title
    • Description of what's being tested
    • Reference to Kubo source
    • Step-by-step verification instructions
    • Evidence of compliance
    • Pass/fail status
  4. Kubo interoperability - Critical requirements explicitly test cross-implementation compatibility

  5. Traceability - Every requirement links to Kubo source code

The compliance report format ensures any human reviewer can:

  • Understand what each requirement specifies
  • See exactly what test validates it
  • Verify the evidence independently
  • Confirm the requirement is met