This document is the controlling specification for the ferripfs project.
| Phase | Name | Status | Compliance Tests | Kubo Tests Ported | Notes |
|---|---|---|---|---|---|
| 1 | Project Foundation | COMPLETE | 10/10 | N/A | Workspace, licenses, docs, CLI skeleton |
| 2 | Configuration and Repository | COMPLETE | 27/27 | 17/25 files | Config, repo, and migration tests ported |
| 3 | Blockstore and Content Addressing | INCOMPLETE | 15/15 | 0/5+ files | Implementation done, Go tests NOT ported |
| 4 | UnixFS and File Operations | INCOMPLETE | 27/27 | 0/6+ files | Implementation done, Go tests NOT ported |
| 5 | Pinning System | INCOMPLETE | 19/19 | 0/3+ files | Implementation done, Go tests NOT ported |
| 6 | DAG Operations | INCOMPLETE | 13/13 | 0/14+ files | Implementation done, Go tests NOT ported |
| 7 | Networking (libp2p) | INCOMPLETE | 12/12 | 0/3 files | Implementation done, Go tests NOT ported |
| 8 | DHT and Routing | NOT STARTED | 0/16 | 0/4+ files | Kademlia, content routing |
| 9 | Bitswap and Content Exchange | NOT STARTED | 0/12 | 0/5+ files | Block exchange protocol |
| 10 | IPNS and Key Management | NOT STARTED | 0/23 | 0/6+ files | IPNS records, key gen |
| 11 | MFS (Mutable File System) | NOT STARTED | 0/20 | 0/3+ files | files subcommand |
| 12 | PubSub | NOT STARTED | 0/10 | 0/4+ files | GossipSub/FloodSub |
| 13 | HTTP Gateway | NOT STARTED | 0/15 | 0/4+ files | Path/subdomain gateway |
| 14 | HTTP RPC API | NOT STARTED | 0/10 | 0/4+ files | JSON-RPC API server |
| 15 | Remaining Commands | NOT STARTED | 0/varies | 0/10+ files | Full CLI parity |
CRITICAL: A phase is NOT COMPLETE until:
- Implementation passes all compliance tests
- ALL relevant Kubo Go tests have been ported and pass
- Port attribution headers reference original Go test files
This requirement applies to ALL phases (2-15). See "Kubo Test Files to Port (Detailed)" section below for the complete list of Go tests that must be ported for each phase.
ferripfs/
├── bin/ferripfs/ # Main CLI binary
├── crates/
│ ├── ferripfs-blockstore/ # Block storage, CID handling
│ ├── ferripfs-config/ # Configuration types
│ ├── ferripfs-dag/ # DAG/IPLD operations
│ ├── ferripfs-network/ # libp2p networking (Phase 7)
│ ├── ferripfs-pinning/ # Pinning system
│ ├── ferripfs-repo/ # Repository management
│ ├── ferripfs-test-harness/ # Testing infrastructure
│ ├── ferripfs-tests/ # Integration tests
│ │ └── tests/
│ │ ├── phase1_compliance.rs # Phase 1 compliance tests
│ │ ├── phase2_compliance.rs # Phase 2 compliance tests
│ │ ├── ... # phaseN_compliance.rs for each phase
│ │ └── kubo_ported/ # PORTED KUBO GO TESTS (REQUIRED)
│ │ ├── phase2_config_test.rs
│ │ ├── phase2_repo_test.rs
│ │ ├── phase3_blockstore_test.rs
│ │ ├── phase4_unixfs_test.rs
│ │ ├── phase5_pinning_test.rs
│ │ ├── phase6_dag_test.rs
│ │ ├── phase7_network_test.rs
│ │ ├── phase8_dht_test.rs
│ │ ├── phase9_bitswap_test.rs
│ │ ├── phase10_ipns_test.rs
│ │ ├── phase11_mfs_test.rs
│ │ ├── phase12_pubsub_test.rs
│ │ ├── phase13_gateway_test.rs
│ │ ├── phase14_rpc_test.rs
│ │ └── phase15_commands_test.rs
│ └── ferripfs-unixfs/ # UnixFS implementation
Goal: Port Kubo (IPFS Go implementation) v0.39.0 to Rust with full feature and CLI command parity.
Project Name: ferripfs
Source: /home/dpp/tmp/rustipfs/kubo (tag v0.39.0)
License: Dual MIT/Apache-2.0 (matching Kubo)
Approach: Port fresh from Kubo for maximum fidelity; use only low-level ecosystem crates (cid, multihash, libp2p)
Principles:
- Ethical, transparent port with explicit attribution
- Every ported file references its Kubo source
- Feature-by-feature and command-by-command verifiable parity
- Rust library + CLI binary
- 85% minimum test coverage per phase
- Human-readable compliance reports from integration tests
- Prefer existing open source implementations (see Library-First Development below)
Never write a module or functionality if there is an existing open source implementation available.
The Rust ecosystem has mature crates for most IPFS-related functionality. Before writing any code, search for and evaluate existing crates that provide the needed functionality.
-
Search Before Writing: Before implementing any module, search crates.io and GitHub for existing implementations. Evaluate:
rust-ipfsecosystem crateslibp2pcrates for networkingcid,multihash,multiaddrfor content addressingprostwith pre-generated code orquick-protobuffor protobuf- Any other well-maintained crates that provide the needed functionality
-
Stop on Missing Dependencies: If a build or runtime dependency is missing (e.g.,
protocnot installed, system library unavailable), STOP immediately and prompt the user to install the required dependency. Do not attempt workarounds like:- Writing hand-coded alternatives
- Generating code manually
- Using different approaches to avoid the dependency
-
Permission Required for Custom Code: If custom code must be written instead of using an existing library, ask the user for explicit permission with a complete justification that includes:
- What libraries were evaluated and why they are unsuitable
- What specific functionality is missing from existing crates
- Why the custom implementation is necessary
- The scope and complexity of the custom code
Scenario: Protobuf compilation requires protoc
- WRONG: Write hand-coded protobuf structures to avoid the dependency
- RIGHT: Stop and tell the user: "Building ferripfs-unixfs requires
protoc. Please install it with:sudo apt-get install protobuf-compiler"
Scenario: Need UnixFS chunking
- FIRST: Search for existing crates (
ipfs-unixfs,rust-ipfsmodules, etc.) - IF NONE SUITABLE: Ask user: "I could not find an existing Rust crate for UnixFS chunking that matches Kubo's implementation. The closest is X but it lacks Y. May I implement custom chunking code?"
Scenario: Need CID handling
- WRONG: Implement CID parsing from scratch
- RIGHT: Use the
cidcrate from the rust-ipfs ecosystem
The following crates are pre-approved for use without further justification:
cid,multihash,multihash-codetable,multiaddr- Content addressinglibp2pand its sub-crates - Networkingprost,prost-build- Protobuf (requiresprotoc)serde,serde_json- Serializationtokio- Async runtimeclap- CLI parsingthiserror,anyhow- Error handling- Standard Rust ecosystem crates for common tasks
Each phase must achieve:
- 85% line coverage measured by
cargo-tarpaulinorllvm-cov - 100% coverage of public API surface
- Integration tests for every requirement
- Ported Kubo Go tests for all relevant functionality
For each phase, the relevant Go tests from Kubo must be identified, ported to Rust, and included as part of the phase's test suite.
-
Identify Relevant Kubo Tests: Before implementing a phase, locate all Go test files in Kubo that test the functionality being ported. Key locations include:
kubo/test/- Integration and CLI testskubo/core/commands/*_test.go- Command testsboxo/*/test/- Component tests (blockstore, unixfs, etc.)- Package-level
*_test.gofiles adjacent to implementation
-
Document Test Mapping: Create a test mapping document for each phase listing:
- Original Go test file path
- Test function names
- Corresponding Rust test location
- Any behavioral differences or limitations
-
Port Tests Faithfully: Tests should be ported to maintain:
- Same test inputs and expected outputs
- Same edge cases and error conditions
- Same behavioral assertions
- Comments referencing the original Go test
-
Test File Naming Convention: Ported tests should follow:
tests/kubo_ported/ ├── phase1_tests.rs ├── phase2_config_test.rs ├── phase2_repo_test.rs └── ... -
Attribution in Ported Tests: Each ported test file must include:
// Ported from: kubo/path/to/original_test.go // Kubo version: v0.39.0 // Original test: TestFunctionName // // Original work: Copyright (c) Protocol Labs, Inc. // Port: Copyright (c) 2026 ferripfs contributors // SPDX-License-Identifier: MIT OR Apache-2.0
The following lists the specific Go test files in Kubo v0.39.0 that MUST be ported for each phase. A phase is not complete until all listed tests are ported.
Config Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/config/api_test.go |
TestConvertAuthSecret | PORTED → phase2_config.rs::test_convert_auth_secret |
kubo/config/autoconf_test.go |
TestAutoConfDefaults, TestAutoConfProfile, TestInitWithAutoValues | DEFERRED (autoconf not implemented) |
kubo/config/bootstrap_peers_test.go |
TestBootstrapPeerStrings | PORTED → phase2_config.rs::test_bootstrap_peer_strings |
kubo/config/config_test.go |
TestClone, TestReflectToMap, TestCheckKey | PORTED → phase2_config.rs::test_clone, test_check_key |
kubo/config/import_test.go |
10 validation tests (HAMTFanout, CidVersion, UnixFSFileMaxLinks, etc.) | DEFERRED (validation API not implemented) |
kubo/config/init_test.go |
TestCreateIdentity, TestCreateIdentityOptions | DEFERRED (CreateIdentity in ferripfs-repo, not config) |
kubo/config/migration_test.go |
TestMigrationDecode | PORTED → phase2_config.rs::test_migration_decode |
kubo/config/provide_test.go |
TestParseProvideStrategy, TestValidateProvideConfig_*, TestShouldProvideForStrategy | DEFERRED (provide strategy not implemented) |
kubo/config/routing_test.go |
TestRouterParameters, TestMethods | PORTED → phase2_config.rs::test_router_parameters_roundtrip |
kubo/config/serialize/serialize_test.go |
TestConfig | DEFERRED (file doesn't exist in kubo) |
kubo/config/types_test.go |
10 type tests (OptionalDuration, Strings, Flag, Priority, Integer, String, Bytes) | PORTED → phase2_config.rs::test_optional_*, test_strings_*, test_flag_*, test_priority_* |
Repository Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/repo/common/common_test.go |
4 map merge tests | PORTED → phase2_repo.rs::test_map_merge_deep_* |
kubo/repo/fsrepo/config_test.go |
4 datastore config tests | DEFERRED (datastore plugin system not implemented) |
kubo/repo/fsrepo/fsrepo_test.go |
5 repo lifecycle tests | PORTED → phase2_repo.rs::test_init_*, test_can_manage_*, test_datastore_*, test_open_* |
Migration Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/repo/fsrepo/migrations/atomicfile/atomicfile_test.go |
9 atomic file tests | PORTED → phase2_migrations.rs::test_atomic_file_* |
kubo/repo/fsrepo/migrations/embedded_test.go |
3 embedded migration tests | PORTED → phase2_migrations.rs::test_*_embedded_migration* |
kubo/repo/fsrepo/migrations/fetch_test.go |
TestGetDistPath, TestMultiFetcher | PORTED → phase2_migrations.rs::test_get_dist_path |
kubo/repo/fsrepo/migrations/ipfsdir_test.go |
TestExpandHome, TestIpfsDir, TestCheckIpfsDir, TestRepoVersion | PORTED → phase2_migrations.rs::test_expand_home_*, test_ipfs_dir_*, test_check_ipfs_dir_*, test_repo_version* |
kubo/repo/fsrepo/migrations/migrations_test.go |
TestFindMigrations, TestMigrationName, TestExeName, TestNeedMigration | PORTED → phase2_migrations.rs::test_find_migrations*, test_migration_name, test_exe_name, test_need_migration |
kubo/repo/fsrepo/migrations/versions_test.go |
TestParseSemver, TestCompareVersions, TestSortVersions | PORTED → phase2_migrations.rs::test_parse_semver, test_compare_versions, test_sort_versions, test_parse_versions_list, test_find_latest* |
Note: Blockstore tests are in the boxo library (github.com/ipfs/boxo), not in kubo repo. Must be obtained from boxo v0.35.2.
| Source | Test Functions | Port Status |
|---|---|---|
boxo/blockstore/blockstore_test.go |
Blockstore CRUD operations | NOT PORTED |
boxo/blockstore/bloom_cache_test.go |
Bloom filter caching tests | NOT PORTED |
boxo/blockstore/caching_test.go |
Caching blockstore tests | NOT PORTED |
go-cid library |
CID v0/v1 parsing and encoding tests | NOT PORTED |
go-multihash library |
Multihash encoding tests | NOT PORTED |
Kubo Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/coreunix/add_test.go |
TestAddMultipleGCLive, TestAddGCLive, TestAddWPosInfo, TestAddWPosInfoAndRawLeafs | NOT PORTED |
kubo/core/coreunix/metadata_test.go |
TestMetadata | NOT PORTED |
Boxo Tests (from boxo library):
| Source | Test Functions | Port Status |
|---|---|---|
boxo/chunker/splitting_test.go |
Size chunker tests | NOT PORTED |
boxo/chunker/rabin_test.go |
Rabin chunker tests | NOT PORTED |
boxo/ipld/unixfs/importer/*_test.go |
DAG builder tests | NOT PORTED |
boxo/ipld/unixfs/unixfs_test.go |
UnixFS encoding tests | NOT PORTED |
Kubo Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/commands/pin/remotepin_test.go |
TestNormalizeEndpoint | NOT PORTED |
Boxo Tests (from boxo library):
| Source | Test Functions | Port Status |
|---|---|---|
boxo/pinning/pinner/pin_test.go |
Pin/unpin operations, recursive pins | NOT PORTED |
boxo/pinning/pinner/dspinner/*_test.go |
Datastore pinner tests | NOT PORTED |
Command Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/commands/cid_test.go |
CID command tests | NOT PORTED |
kubo/core/commands/commands_test.go |
Command structure tests | NOT PORTED |
kubo/core/commands/config_test.go |
Config command tests | NOT PORTED |
kubo/core/commands/dht_test.go |
DHT command tests | NOT PORTED |
kubo/core/commands/files_test.go |
Files command tests | NOT PORTED |
kubo/core/commands/get_test.go |
Get command tests | NOT PORTED |
kubo/core/commands/helptext_test.go |
Help text tests | NOT PORTED |
kubo/core/commands/repo_verify_test.go |
Repo verify tests | NOT PORTED |
kubo/core/commands/root_test.go |
Root command tests | NOT PORTED |
kubo/core/commands/cidbase_test.go |
CID base tests | NOT PORTED |
kubo/core/commands/env_test.go |
Environment tests | NOT PORTED |
kubo/core/commands/utils_test.go |
TestPathOrCidPath, TestValidatePinName | NOT PORTED |
Boxo Tests (from boxo library):
| Source | Test Functions | Port Status |
|---|---|---|
boxo/ipld/merkledag/*_test.go |
MerkleDAG tests | NOT PORTED |
boxo/ipld/car/*_test.go |
CAR format tests | NOT PORTED |
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/node/libp2p/libp2p_test.go |
TestPrioritize | NOT PORTED |
kubo/core/node/libp2p/rcmgr_logging_test.go |
TestLoggingResourceManager | NOT PORTED |
kubo/core/node/libp2p/routingopt_test.go |
TestHttpAddrsFromConfig, TestDetermineCapabilities, TestEndpointCapabilitiesReadWriteLogic | NOT PORTED |
Kubo Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/commands/dht_test.go |
DHT command tests | NOT PORTED |
kubo/routing/*_test.go |
Routing tests | NOT PORTED |
External Library Tests (go-libp2p-kad-dht):
| Source | Test Functions | Port Status |
|---|---|---|
go-libp2p-kad-dht/*_test.go |
Kademlia DHT tests | NOT PORTED |
go-libp2p-kad-dht/routing_test.go |
Routing table tests | NOT PORTED |
Boxo Tests:
| Source | Test Functions | Port Status |
|---|---|---|
boxo/bitswap/bitswap_test.go |
Core bitswap tests | NOT PORTED |
boxo/bitswap/client/*_test.go |
Bitswap client tests | NOT PORTED |
boxo/bitswap/server/*_test.go |
Bitswap server tests | NOT PORTED |
boxo/bitswap/wantlist/*_test.go |
Wantlist tests | NOT PORTED |
boxo/bitswap/decision/*_test.go |
Decision engine tests | NOT PORTED |
Kubo Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/commands/keystore_test.go |
Keystore command tests | NOT PORTED |
kubo/core/commands/name/*_test.go |
Name command tests | NOT PORTED |
Boxo Tests:
| Source | Test Functions | Port Status |
|---|---|---|
boxo/ipns/*_test.go |
IPNS record tests | NOT PORTED |
boxo/namesys/*_test.go |
Name system resolver tests | NOT PORTED |
boxo/namesys/publisher_test.go |
IPNS publisher tests | NOT PORTED |
boxo/keystore/*_test.go |
Key storage tests | NOT PORTED |
Kubo Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/commands/files_test.go |
Files command tests | NOT PORTED |
Boxo Tests:
| Source | Test Functions | Port Status |
|---|---|---|
boxo/mfs/*_test.go |
MFS directory/file tests | NOT PORTED |
boxo/mfs/mfs_test.go |
Core MFS operations | NOT PORTED |
External Library Tests (go-libp2p-pubsub):
| Source | Test Functions | Port Status |
|---|---|---|
go-libp2p-pubsub/*_test.go |
PubSub tests | NOT PORTED |
go-libp2p-pubsub/gossipsub_test.go |
GossipSub tests | NOT PORTED |
go-libp2p-pubsub/floodsub_test.go |
FloodSub tests | NOT PORTED |
Kubo Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/commands/pubsub_test.go |
PubSub command tests | NOT PORTED |
Boxo Tests:
| Source | Test Functions | Port Status |
|---|---|---|
boxo/gateway/*_test.go |
Gateway handler tests | NOT PORTED |
boxo/gateway/gateway_test.go |
Core gateway tests | NOT PORTED |
boxo/gateway/handler_test.go |
Request handler tests | NOT PORTED |
boxo/gateway/metrics_test.go |
Gateway metrics tests | NOT PORTED |
Kubo Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/corehttp/*_test.go |
HTTP API tests | NOT PORTED |
kubo/core/corehttp/corehttp_test.go |
Core HTTP tests | NOT PORTED |
kubo/core/corehttp/commands_test.go |
Command API tests | NOT PORTED |
kubo/client/rpc/*_test.go |
RPC client tests | NOT PORTED |
Kubo Tests:
| File | Test Functions | Port Status |
|---|---|---|
kubo/core/commands/diag_test.go |
Diagnostic command tests | NOT PORTED |
kubo/core/commands/log_test.go |
Log command tests | NOT PORTED |
kubo/core/commands/mount_test.go |
Mount command tests | NOT PORTED |
kubo/core/commands/p2p_test.go |
P2P command tests | NOT PORTED |
kubo/core/commands/stat_test.go |
Stat command tests | NOT PORTED |
kubo/core/commands/swarm_test.go |
Swarm command tests | NOT PORTED |
kubo/core/commands/version_test.go |
Version command tests | NOT PORTED |
kubo/test/cli/*_test.go |
CLI integration tests | NOT PORTED |
Kubo includes a comprehensive sharness-based CLI test suite in kubo/test/sharness/. These shell-based tests should also be ported:
- Location:
kubo/test/sharness/t*.sh - Porting Strategy: Convert to Rust integration tests using
Commandto invoke the CLI - Priority: Port sharness tests that cover functionality not already tested by unit tests
Integration tests produce a JSON report and human-readable summary:
================================================================================
FERRIPFS PHASE [N] COMPLIANCE REPORT
Generated: [timestamp]
Kubo Reference: v0.39.0
================================================================================
REQUIREMENT: [REQ-N.M] [Requirement Title]
------------------------------------------------------------------------------
Description: [What this requirement specifies]
Kubo Source: [Path to reference Go file]
Test: [test_name]
Status: PASS | FAIL
How to Verify:
[Step-by-step instructions for human verification]
Evidence:
[Captured output, file contents, or observable behavior]
------------------------------------------------------------------------------
PHASE SUMMARY
------------------------------------------------------------------------------
Total Requirements: [X]
Passed: [Y]
Failed: [Z]
Coverage: [XX.X%]
Status: COMPLIANT | NON-COMPLIANT
================================================================================
At the completion of each phase, the implementer MUST provide a detailed compliance demonstration that includes:
- Individual Requirement Listing: Each requirement must be listed separately with its ID and title
- Test Validation: For each requirement, identify the specific test function that validates it
- Human Verification Guide: For each requirement, provide clear instructions explaining how a human reviewer can independently verify compliance based on the test output
The demonstration format for each requirement:
### REQ-X.Y: [Requirement Title]
**Test**: `test_function_name` in `path/to/test_file.rs`
**Status**: PASS | FAIL
**Evidence from test output**:
[Relevant excerpt from the test output showing the validation]
**How to verify manually**:
1. [Step-by-step instructions]
2. [What to look for in the output]
3. [Expected values or patterns that confirm compliance]
This ensures:
- Complete traceability from requirement to test to evidence
- Any reviewer can independently confirm each requirement was met
- The demonstration serves as living documentation of compliance
Create ferripfs-test-harness crate providing:
/// A requirement that must be tested
pub struct Requirement {
pub id: String, // e.g., "REQ-1.3"
pub title: String,
pub description: String,
pub kubo_source: String, // e.g., "kubo/config/config.go:45-120"
pub verification_steps: Vec<String>,
}
/// Result of testing a requirement
pub struct RequirementResult {
pub requirement: Requirement,
pub test_name: String,
pub status: TestStatus,
pub evidence: String,
pub duration: Duration,
}
/// Generate human-readable compliance report
pub fn generate_compliance_report(phase: u8, results: Vec<RequirementResult>) -> String;| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-1.1 | Cargo workspace compiles with no errors | N/A | Build |
| REQ-1.2 | LICENSE-MIT matches Kubo LICENSE-MIT exactly | LICENSE-MIT |
Unit |
| REQ-1.3 | LICENSE-APACHE matches Kubo LICENSE-APACHE exactly | LICENSE-APACHE |
Unit |
| REQ-1.4 | README.md states this is a port of Kubo v0.39.0 | N/A | Unit |
| REQ-1.5 | ATTRIBUTION.md acknowledges Kubo, Protocol Labs, IPFS contributors | N/A | Unit |
| REQ-1.6 | PORTING_GUIDE.md specifies file header format | N/A | Unit |
| REQ-1.7 | PARITY.md lists all 131 Kubo commands with status tracking | core/commands/root.go |
Unit |
| REQ-1.8 | Test harness can generate compliance reports | N/A | Integration |
| REQ-1.9 | ferripfs --version outputs version string |
cmd/ipfs/kubo/version.go |
Integration |
| REQ-1.10 | ferripfs --help outputs help text with command list |
core/commands/root.go |
Integration |
ferripfs/
├── Cargo.toml
├── LICENSE-MIT
├── LICENSE-APACHE
├── README.md
├── ATTRIBUTION.md
├── PORTING_GUIDE.md
├── PARITY.md
├── crates/
│ └── ferripfs-test-harness/
│ ├── Cargo.toml
│ └── src/
│ ├── lib.rs
│ ├── requirement.rs
│ ├── report.rs
│ └── runner.rs
├── bin/
│ └── ferripfs/
│ ├── Cargo.toml
│ └── src/
│ └── main.rs
└── tests/
└── phase1/
├── compliance.rs
└── fixtures/
#[test]
fn req_1_2_license_mit_matches_kubo() {
let req = Requirement {
id: "REQ-1.2".into(),
title: "LICENSE-MIT matches Kubo".into(),
description: "The LICENSE-MIT file must be byte-for-byte identical to Kubo's LICENSE-MIT".into(),
kubo_source: "LICENSE-MIT".into(),
verification_steps: vec![
"Read ferripfs/LICENSE-MIT".into(),
"Read kubo/LICENSE-MIT".into(),
"Compare byte-for-byte".into(),
"Files must be identical".into(),
],
};
let ferripfs_license = fs::read_to_string("LICENSE-MIT").unwrap();
let kubo_license = fs::read_to_string("../kubo/LICENSE-MIT").unwrap();
let status = if ferripfs_license == kubo_license {
TestStatus::Pass
} else {
TestStatus::Fail
};
let evidence = format!(
"ferripfs LICENSE-MIT length: {} bytes\n\
kubo LICENSE-MIT length: {} bytes\n\
Match: {}",
ferripfs_license.len(),
kubo_license.len(),
ferripfs_license == kubo_license
);
record_result(RequirementResult {
requirement: req,
test_name: "req_1_2_license_mit_matches_kubo".into(),
status,
evidence,
duration: elapsed,
});
}#[test]
fn req_1_7_parity_lists_all_commands() {
let req = Requirement {
id: "REQ-1.7".into(),
title: "PARITY.md lists all Kubo commands".into(),
description: "PARITY.md must list all 131 Kubo CLI commands with tracking status".into(),
kubo_source: "core/commands/root.go".into(),
verification_steps: vec![
"Parse Kubo core/commands/root.go for command list".into(),
"Parse PARITY.md for listed commands".into(),
"Every Kubo command must appear in PARITY.md".into(),
"Each entry must have status indicator (not started/in progress/complete)".into(),
],
};
let kubo_commands = extract_kubo_commands("../kubo/core/commands/");
let parity_commands = parse_parity_md("PARITY.md");
let missing: Vec<_> = kubo_commands.iter()
.filter(|c| !parity_commands.contains(c))
.collect();
let status = if missing.is_empty() { TestStatus::Pass } else { TestStatus::Fail };
let evidence = format!(
"Kubo commands found: {}\n\
PARITY.md entries: {}\n\
Missing commands: {:?}",
kubo_commands.len(),
parity_commands.len(),
missing
);
// ... record result
}#[test]
fn req_1_9_version_command() {
let req = Requirement {
id: "REQ-1.9".into(),
title: "Version command outputs version string".into(),
description: "ferripfs --version must output version in format 'ferripfs version X.Y.Z (kubo v0.39.0 port)'".into(),
kubo_source: "cmd/ipfs/kubo/version.go".into(),
verification_steps: vec![
"Run: ferripfs --version".into(),
"Output must contain 'ferripfs version'".into(),
"Output must contain 'kubo v0.39.0 port'".into(),
"Exit code must be 0".into(),
],
};
let output = Command::new("./target/debug/ferripfs")
.arg("--version")
.output()
.unwrap();
let stdout = String::from_utf8_lossy(&output.stdout);
let has_version = stdout.contains("ferripfs version");
let has_kubo_ref = stdout.contains("kubo v0.39.0 port");
let exit_ok = output.status.success();
let status = if has_version && has_kubo_ref && exit_ok {
TestStatus::Pass
} else {
TestStatus::Fail
};
let evidence = format!(
"Command: ferripfs --version\n\
Exit code: {}\n\
stdout:\n{}\n\
Contains 'ferripfs version': {}\n\
Contains 'kubo v0.39.0 port': {}",
output.status.code().unwrap_or(-1),
stdout,
has_version,
has_kubo_ref
);
// ... record result
}Run: cargo test --test phase1_compliance -- --nocapture
Output:
================================================================================
FERRIPFS PHASE 1 COMPLIANCE REPORT
Generated: 2024-XX-XX HH:MM:SS UTC
Kubo Reference: v0.39.0
================================================================================
REQUIREMENT: [REQ-1.1] Cargo workspace compiles
------------------------------------------------------------------------------
Description: Cargo workspace must compile with no errors or warnings
Kubo Source: N/A
Test: req_1_1_workspace_compiles
Status: PASS
How to Verify:
1. Run: cargo build --workspace
2. Observe zero errors
3. Observe zero warnings (with -D warnings)
Evidence:
cargo build exit code: 0
Errors: 0
Warnings: 0
------------------------------------------------------------------------------
REQUIREMENT: [REQ-1.2] LICENSE-MIT matches Kubo
------------------------------------------------------------------------------
Description: The LICENSE-MIT file must be byte-for-byte identical to Kubo's LICENSE-MIT
Kubo Source: LICENSE-MIT
Test: req_1_2_license_mit_matches_kubo
Status: PASS
How to Verify:
1. Read ferripfs/LICENSE-MIT
2. Read kubo/LICENSE-MIT
3. Compare byte-for-byte
4. Files must be identical
Evidence:
ferripfs LICENSE-MIT length: 1076 bytes
kubo LICENSE-MIT length: 1076 bytes
Match: true
------------------------------------------------------------------------------
[... continues for all requirements ...]
PHASE 1 SUMMARY
------------------------------------------------------------------------------
Total Requirements: 10
Passed: 10
Failed: 0
Coverage: 87.3%
Status: COMPLIANT
================================================================================
| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-2.1 | Config struct deserializes Kubo config.json | config/config.go |
Unit |
| REQ-2.2 | Config struct serializes to Kubo-compatible JSON | config/config.go |
Unit |
| REQ-2.3 | Identity config section: PeerID, PrivKey | config/identity.go |
Unit |
| REQ-2.4 | Datastore config section: Spec, Type | config/datastore.go |
Unit |
| REQ-2.5 | Addresses config: Swarm, API, Gateway, Announce, NoAnnounce | config/addresses.go |
Unit |
| REQ-2.6 | Bootstrap config: list of multiaddrs | config/bootstrap_peers.go |
Unit |
| REQ-2.7 | Swarm config: ConnMgr, Transports, RelayClient | config/swarm.go |
Unit |
| REQ-2.8 | Routing config: Type, Routers | config/routing.go |
Unit |
| REQ-2.9 | FSRepo initializes directory structure | repo/fsrepo/fsrepo.go |
Integration |
| REQ-2.10 | FSRepo creates config file | repo/fsrepo/fsrepo.go |
Integration |
| REQ-2.11 | FSRepo creates datastore directory | repo/fsrepo/fsrepo.go |
Integration |
| REQ-2.12 | FSRepo creates blocks directory | repo/fsrepo/fsrepo.go |
Integration |
| REQ-2.13 | FSRepo version file contains "18" | repo/fsrepo/fsrepo.go:28 |
Integration |
| REQ-2.14 | FSRepo lock prevents concurrent access | repo/fsrepo/lock.go |
Integration |
| REQ-2.15 | Keystore stores Ed25519 private keys | repo/fsrepo/keystore/keystore.go |
Unit |
| REQ-2.16 | Keystore stores RSA private keys | repo/fsrepo/keystore/keystore.go |
Unit |
| REQ-2.17 | ferripfs init creates valid repo |
cmd/ipfs/kubo/init.go |
Integration |
| REQ-2.18 | ferripfs init generates peer identity |
cmd/ipfs/kubo/init.go |
Integration |
| REQ-2.19 | ferripfs init --profile=server applies profile |
config/profile.go |
Integration |
| REQ-2.20 | ferripfs config show outputs JSON config |
core/commands/config.go |
Integration |
| REQ-2.21 | ferripfs config <key> gets config value |
core/commands/config.go |
Integration |
| REQ-2.22 | ferripfs config <key> <value> sets config value |
core/commands/config.go |
Integration |
| REQ-2.23 | ferripfs config replace replaces config file |
core/commands/config.go |
Integration |
| REQ-2.24 | Config migration from Kubo repo works | N/A | Integration |
| REQ-2.25 | Datastore interface: Put, Get, Has, Delete, Query | repo/repo.go |
Unit |
| REQ-2.26 | FlatFS datastore implementation | go-ds-flatfs |
Integration |
| REQ-2.27 | Datastore batching support | go-datastore |
Unit |
crates/
├── ferripfs-config/
│ ├── Cargo.toml
│ └── src/
│ ├── lib.rs
│ ├── config.rs # Main Config struct
│ ├── identity.rs # Identity section
│ ├── datastore.rs # Datastore section
│ ├── addresses.rs # Addresses section
│ ├── bootstrap.rs # Bootstrap peers
│ ├── swarm.rs # Swarm settings
│ ├── routing.rs # Routing config
│ ├── gateway.rs # Gateway settings
│ ├── api.rs # API settings
│ ├── profile.rs # Config profiles
│ └── tests/
│ ├── kubo_compat.rs # Test against real Kubo configs
│ └── fixtures/
│ └── kubo_config.json
├── ferripfs-repo/
│ ├── Cargo.toml
│ └── src/
│ ├── lib.rs
│ ├── repo.rs # Repo trait
│ ├── fsrepo.rs # Filesystem repo
│ ├── lock.rs # Repo locking
│ ├── keystore.rs # Key storage
│ ├── datastore/
│ │ ├── mod.rs
│ │ ├── traits.rs # Datastore trait
│ │ ├── flatfs.rs # FlatFS implementation
│ │ ├── memory.rs # In-memory (for tests)
│ │ └── batch.rs # Batching support
│ └── version.rs # Repo versioning
#[test]
fn req_2_1_deserialize_kubo_config() {
let req = Requirement {
id: "REQ-2.1".into(),
title: "Config struct deserializes Kubo config.json".into(),
description: "ferripfs Config must successfully parse a config.json generated by Kubo".into(),
kubo_source: "config/config.go".into(),
verification_steps: vec![
"Generate config.json using: kubo init --repo=/tmp/kubo-test".into(),
"Read /tmp/kubo-test/config".into(),
"Deserialize using ferripfs_config::Config::from_reader()".into(),
"All fields must parse without error".into(),
"Identity.PeerID must be valid PeerId".into(),
"Addresses.Swarm must contain at least one multiaddr".into(),
],
};
// Generate Kubo config
let kubo_repo = tempdir().unwrap();
Command::new("ipfs")
.args(["init", "--repo", kubo_repo.path().to_str().unwrap()])
.output()
.expect("Kubo must be installed for interop tests");
let config_path = kubo_repo.path().join("config");
let config_json = fs::read_to_string(&config_path).unwrap();
let result = Config::from_str(&config_json);
let status = match &result {
Ok(config) => {
let peer_id_valid = config.identity.peer_id.parse::<PeerId>().is_ok();
let has_swarm_addr = !config.addresses.swarm.is_empty();
if peer_id_valid && has_swarm_addr {
TestStatus::Pass
} else {
TestStatus::Fail
}
}
Err(_) => TestStatus::Fail,
};
let evidence = match &result {
Ok(config) => format!(
"Kubo config parsed successfully\n\
Config file size: {} bytes\n\
PeerID: {}\n\
Swarm addresses: {:?}\n\
API address: {:?}\n\
Gateway address: {:?}\n\
Bootstrap peers: {} entries",
config_json.len(),
config.identity.peer_id,
config.addresses.swarm,
config.addresses.api,
config.addresses.gateway,
config.bootstrap.len()
),
Err(e) => format!("Parse error: {}", e),
};
// ... record result
}#[test]
fn req_2_17_init_creates_valid_repo() {
let req = Requirement {
id: "REQ-2.17".into(),
title: "ferripfs init creates valid repository".into(),
description: "Running 'ferripfs init' must create a complete, valid IPFS repository".into(),
kubo_source: "cmd/ipfs/kubo/init.go".into(),
verification_steps: vec![
"Create empty temp directory".into(),
"Run: IPFS_PATH=/tmp/test ferripfs init".into(),
"Verify exit code is 0".into(),
"Verify /tmp/test/config exists and is valid JSON".into(),
"Verify /tmp/test/datastore directory exists".into(),
"Verify /tmp/test/blocks directory exists".into(),
"Verify /tmp/test/version contains '18'".into(),
"Verify config contains valid Identity.PeerID".into(),
"Verify generated PeerID matches Identity.PrivKey".into(),
],
};
let repo_path = tempdir().unwrap();
let repo_str = repo_path.path().to_str().unwrap();
let output = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", repo_str)
.arg("init")
.output()
.unwrap();
let config_exists = repo_path.path().join("config").exists();
let datastore_exists = repo_path.path().join("datastore").is_dir();
let blocks_exists = repo_path.path().join("blocks").is_dir();
let version_content = fs::read_to_string(repo_path.path().join("version"))
.unwrap_or_default();
let config_valid = if config_exists {
let config_str = fs::read_to_string(repo_path.path().join("config")).unwrap();
Config::from_str(&config_str).is_ok()
} else {
false
};
let all_pass = output.status.success()
&& config_exists
&& config_valid
&& datastore_exists
&& blocks_exists
&& version_content.trim() == "18";
let status = if all_pass { TestStatus::Pass } else { TestStatus::Fail };
let evidence = format!(
"Command: IPFS_PATH={} ferripfs init\n\
Exit code: {}\n\
stdout: {}\n\
stderr: {}\n\
\n\
Repository structure:\n\
- config exists: {} (valid JSON: {})\n\
- datastore/ exists: {}\n\
- blocks/ exists: {}\n\
- version content: '{}'",
repo_str,
output.status.code().unwrap_or(-1),
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr),
config_exists,
config_valid,
datastore_exists,
blocks_exists,
version_content.trim()
);
// ... record result
}#[test]
fn req_2_24_kubo_repo_migration() {
let req = Requirement {
id: "REQ-2.24".into(),
title: "Config migration from existing Kubo repo".into(),
description: "ferripfs must be able to use an existing Kubo-initialized repository".into(),
kubo_source: "N/A".into(),
verification_steps: vec![
"Initialize repo with Kubo: ipfs init".into(),
"Point ferripfs at same repo: IPFS_PATH=<kubo-repo>".into(),
"Run: ferripfs config show".into(),
"Config must parse and display correctly".into(),
"Run: ferripfs id".into(),
"PeerID must match Kubo's PeerID".into(),
],
};
// Initialize with Kubo
let repo_path = tempdir().unwrap();
let repo_str = repo_path.path().to_str().unwrap();
Command::new("ipfs")
.env("IPFS_PATH", repo_str)
.arg("init")
.output()
.expect("Kubo required");
// Get Kubo's peer ID
let kubo_id = Command::new("ipfs")
.env("IPFS_PATH", repo_str)
.args(["config", "Identity.PeerID"])
.output()
.unwrap();
let kubo_peer_id = String::from_utf8_lossy(&kubo_id.stdout).trim().to_string();
// Use ferripfs with same repo
let ferripfs_config = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", repo_str)
.args(["config", "show"])
.output()
.unwrap();
let ferripfs_id = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", repo_str)
.arg("id")
.output()
.unwrap();
let config_parsed = serde_json::from_slice::<serde_json::Value>(&ferripfs_config.stdout).is_ok();
let id_output = String::from_utf8_lossy(&ferripfs_id.stdout);
let id_matches = id_output.contains(&kubo_peer_id);
let status = if config_parsed && id_matches && ferripfs_config.status.success() {
TestStatus::Pass
} else {
TestStatus::Fail
};
let evidence = format!(
"Kubo repository: {}\n\
Kubo PeerID: {}\n\
\n\
ferripfs config show:\n\
Exit code: {}\n\
Valid JSON: {}\n\
\n\
ferripfs id:\n\
Exit code: {}\n\
Contains Kubo PeerID: {}\n\
Output: {}",
repo_str,
kubo_peer_id,
ferripfs_config.status.code().unwrap_or(-1),
config_parsed,
ferripfs_id.status.code().unwrap_or(-1),
id_matches,
id_output
);
// ... record result
}| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-3.1 | Blockstore trait: Get, Put, Has, Delete, AllKeysChan | boxo/blockstore/blockstore.go |
Unit |
| REQ-3.2 | Block identified by CID | go-block-format |
Unit |
| REQ-3.3 | CIDv0 (sha2-256, dag-pb) support | go-cid |
Unit |
| REQ-3.4 | CIDv1 (multiple codecs, hashes) support | go-cid |
Unit |
| REQ-3.5 | Blockstore persists to FlatFS datastore | boxo/blockstore |
Integration |
| REQ-3.6 | Blockstore retrieves persisted blocks | boxo/blockstore |
Integration |
| REQ-3.7 | GCBlockstore prevents deletion during pin | boxo/blockstore/gcblocker.go |
Unit |
| REQ-3.8 | Block hashing matches Kubo for same content | N/A | Integration |
| REQ-3.9 | ferripfs block put stores raw block |
core/commands/block.go |
Integration |
| REQ-3.10 | ferripfs block get <cid> retrieves block |
core/commands/block.go |
Integration |
| REQ-3.11 | ferripfs block stat <cid> shows size |
core/commands/block.go |
Integration |
| REQ-3.12 | ferripfs block rm <cid> removes block |
core/commands/block.go |
Integration |
| REQ-3.13 | Block put with --format=raw uses raw codec | core/commands/block.go |
Integration |
| REQ-3.14 | Block put with --mhtype=sha2-512 uses sha2-512 | core/commands/block.go |
Integration |
| REQ-3.15 | Caching blockstore reduces disk reads | boxo/blockstore/caching.go |
Unit |
#[test]
fn req_3_8_block_hash_matches_kubo() {
let req = Requirement {
id: "REQ-3.8".into(),
title: "Block hashing matches Kubo".into(),
description: "Given identical content, ferripfs must produce the same CID as Kubo".into(),
kubo_source: "N/A (interoperability requirement)".into(),
verification_steps: vec![
"Create test file with known content".into(),
"Add to Kubo: echo 'hello world' | ipfs block put".into(),
"Add to ferripfs: echo 'hello world' | ferripfs block put".into(),
"CIDs must be identical".into(),
"Repeat with various content sizes: 0 bytes, 1 byte, 256KB, 1MB".into(),
],
};
let test_cases = vec![
("empty", b"".to_vec()),
("hello", b"hello world".to_vec()),
("newline", b"hello world\n".to_vec()),
("256kb", vec![0x42u8; 256 * 1024]),
("1mb", vec![0xFFu8; 1024 * 1024]),
];
let mut results = Vec::new();
for (name, content) in &test_cases {
// Get CID from Kubo
let kubo = Command::new("ipfs")
.args(["block", "put", "--format=raw"])
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.unwrap();
kubo.stdin.unwrap().write_all(content).unwrap();
let kubo_cid = String::from_utf8_lossy(&kubo.wait_with_output().unwrap().stdout)
.trim().to_string();
// Get CID from ferripfs
let ferripfs = Command::new("./target/debug/ferripfs")
.args(["block", "put", "--format=raw"])
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.unwrap();
ferripfs.stdin.unwrap().write_all(content).unwrap();
let ferripfs_cid = String::from_utf8_lossy(&ferripfs.wait_with_output().unwrap().stdout)
.trim().to_string();
results.push((name, content.len(), kubo_cid.clone(), ferripfs_cid.clone(), kubo_cid == ferripfs_cid));
}
let all_match = results.iter().all(|(_, _, _, _, matches)| *matches);
let status = if all_match { TestStatus::Pass } else { TestStatus::Fail };
let evidence = results.iter()
.map(|(name, size, kubo, ferripfs, matches)| {
format!(
"Test '{}' ({} bytes):\n Kubo CID: {}\n ferripfs CID: {}\n Match: {}",
name, size, kubo, ferripfs, matches
)
})
.collect::<Vec<_>>()
.join("\n\n");
// ... record result
}| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-4.1 | Chunker: size-based (default 256KB) | boxo/chunker/splitting.go |
Unit |
| REQ-4.2 | Chunker: rabin fingerprinting | boxo/chunker/rabin.go |
Unit |
| REQ-4.3 | DAG builder: balanced layout | boxo/ipld/unixfs/importer/balanced |
Unit |
| REQ-4.4 | DAG builder: trickle layout | boxo/ipld/unixfs/importer/trickle |
Unit |
| REQ-4.5 | UnixFS protobuf encoding matches Kubo | boxo/ipld/unixfs/pb |
Unit |
| REQ-4.6 | File node type encoding | boxo/ipld/unixfs/unixfs.go |
Unit |
| REQ-4.7 | Directory node type encoding | boxo/ipld/unixfs/unixfs.go |
Unit |
| REQ-4.8 | Symlink node type encoding | boxo/ipld/unixfs/unixfs.go |
Unit |
| REQ-4.9 | HAMT sharded directories (>1000 entries) | boxo/ipld/unixfs/hamt |
Integration |
| REQ-4.10 | Raw leaves mode (--raw-leaves) | core/commands/add.go |
Integration |
| REQ-4.11 | CID version selection (--cid-version) | core/commands/add.go |
Integration |
| REQ-4.12 | ferripfs add <file> returns CID |
core/commands/add.go |
Integration |
| REQ-4.13 | ferripfs add -r <dir> adds directory recursively |
core/commands/add.go |
Integration |
| REQ-4.14 | ferripfs add --wrap-with-directory wraps file |
core/commands/add.go |
Integration |
| REQ-4.15 | ferripfs add --pin=false does not pin |
core/commands/add.go |
Integration |
| REQ-4.16 | ferripfs add --progress shows progress |
core/commands/add.go |
Integration |
| REQ-4.17 | ferripfs add CID matches Kubo for same file |
N/A | Integration |
| REQ-4.18 | ferripfs cat <cid> outputs file content |
core/commands/cat.go |
Integration |
| REQ-4.19 | ferripfs cat --offset --length partial read |
core/commands/cat.go |
Integration |
| REQ-4.20 | ferripfs get <cid> downloads to file |
core/commands/get.go |
Integration |
| REQ-4.21 | ferripfs get -o <path> specifies output |
core/commands/get.go |
Integration |
| REQ-4.22 | ferripfs ls <cid> lists directory entries |
core/commands/ls.go |
Integration |
| REQ-4.23 | ferripfs ls --resolve-type resolves types |
core/commands/ls.go |
Integration |
| REQ-4.24 | ferripfs refs <cid> lists block references |
core/commands/refs.go |
Integration |
| REQ-4.25 | ferripfs refs -r lists recursively |
core/commands/refs.go |
Integration |
| REQ-4.26 | Content added by Kubo can be read by ferripfs | N/A | Integration |
| REQ-4.27 | Content added by ferripfs can be read by Kubo | N/A | Integration |
#[test]
fn req_4_17_add_cid_matches_kubo() {
let req = Requirement {
id: "REQ-4.17".into(),
title: "ferripfs add produces same CID as Kubo".into(),
description: "Adding identical files must produce identical CIDs".into(),
kubo_source: "core/commands/add.go".into(),
verification_steps: vec![
"Create test files of various sizes and types".into(),
"Add each file to Kubo: ipfs add <file>".into(),
"Add each file to ferripfs: ferripfs add <file>".into(),
"Compare CIDs - must be identical".into(),
"Test with: empty file, small text, binary, large file, directory".into(),
],
};
let test_cases = vec![
("empty.txt", vec![]),
("hello.txt", b"hello world\n".to_vec()),
("binary.bin", (0u8..=255).collect()),
("large.bin", vec![0x42u8; 1024 * 1024]), // 1MB
];
let kubo_repo = tempdir().unwrap();
let ferripfs_repo = tempdir().unwrap();
// Initialize repos
init_kubo_repo(&kubo_repo);
init_ferripfs_repo(&ferripfs_repo);
let mut results = Vec::new();
for (name, content) in &test_cases {
let file_path = kubo_repo.path().join(name);
fs::write(&file_path, content).unwrap();
let kubo_output = Command::new("ipfs")
.env("IPFS_PATH", kubo_repo.path())
.args(["add", "-q", file_path.to_str().unwrap()])
.output()
.unwrap();
let kubo_cid = String::from_utf8_lossy(&kubo_output.stdout).trim().to_string();
let ferripfs_output = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", ferripfs_repo.path())
.args(["add", "-q", file_path.to_str().unwrap()])
.output()
.unwrap();
let ferripfs_cid = String::from_utf8_lossy(&ferripfs_output.stdout).trim().to_string();
results.push((name.to_string(), content.len(), kubo_cid.clone(), ferripfs_cid.clone(), kubo_cid == ferripfs_cid));
}
// Directory test
let dir_path = kubo_repo.path().join("testdir");
fs::create_dir(&dir_path).unwrap();
fs::write(dir_path.join("a.txt"), b"file a").unwrap();
fs::write(dir_path.join("b.txt"), b"file b").unwrap();
let kubo_dir = Command::new("ipfs")
.env("IPFS_PATH", kubo_repo.path())
.args(["add", "-rq", dir_path.to_str().unwrap()])
.output()
.unwrap();
let kubo_dir_cid = String::from_utf8_lossy(&kubo_dir.stdout)
.lines().last().unwrap_or("").to_string();
let ferripfs_dir = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", ferripfs_repo.path())
.args(["add", "-rq", dir_path.to_str().unwrap()])
.output()
.unwrap();
let ferripfs_dir_cid = String::from_utf8_lossy(&ferripfs_dir.stdout)
.lines().last().unwrap_or("").to_string();
results.push(("directory".into(), 0, kubo_dir_cid.clone(), ferripfs_dir_cid.clone(), kubo_dir_cid == ferripfs_dir_cid));
let all_match = results.iter().all(|(_, _, _, _, m)| *m);
let status = if all_match { TestStatus::Pass } else { TestStatus::Fail };
let evidence = format!(
"CID Comparison Results:\n\n{}\n\nAll CIDs match: {}",
results.iter()
.map(|(name, size, kubo, ferripfs, matches)| {
format!(
"{}:\n Size: {} bytes\n Kubo: {}\n ferripfs: {}\n Match: {}",
name, size, kubo, ferripfs, if *matches { "YES" } else { "NO" }
)
})
.collect::<Vec<_>>()
.join("\n\n"),
all_match
);
// ... record result
}#[test]
fn req_4_26_27_cross_node_content_exchange() {
let req = Requirement {
id: "REQ-4.26/27".into(),
title: "Cross-node content interoperability".into(),
description: "Content added by one implementation must be readable by the other".into(),
kubo_source: "N/A (interoperability requirement)".into(),
verification_steps: vec![
"Add file with Kubo, read with ferripfs (shared repo)".into(),
"Add file with ferripfs, read with Kubo (shared repo)".into(),
"Compare retrieved content with original".into(),
"Test with various file sizes".into(),
],
};
let shared_repo = tempdir().unwrap();
init_kubo_repo(&shared_repo);
// Test 1: Kubo adds, ferripfs reads
let content1 = b"Content added by Kubo\n";
let file1 = shared_repo.path().join("kubo_file.txt");
fs::write(&file1, content1).unwrap();
let kubo_add = Command::new("ipfs")
.env("IPFS_PATH", shared_repo.path())
.args(["add", "-q", file1.to_str().unwrap()])
.output()
.unwrap();
let cid1 = String::from_utf8_lossy(&kubo_add.stdout).trim().to_string();
let ferripfs_cat = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", shared_repo.path())
.args(["cat", &cid1])
.output()
.unwrap();
let retrieved1 = ferripfs_cat.stdout.clone();
// Test 2: ferripfs adds, Kubo reads
let content2 = b"Content added by ferripfs\n";
let file2 = shared_repo.path().join("ferripfs_file.txt");
fs::write(&file2, content2).unwrap();
let ferripfs_add = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", shared_repo.path())
.args(["add", "-q", file2.to_str().unwrap()])
.output()
.unwrap();
let cid2 = String::from_utf8_lossy(&ferripfs_add.stdout).trim().to_string();
let kubo_cat = Command::new("ipfs")
.env("IPFS_PATH", shared_repo.path())
.args(["cat", &cid2])
.output()
.unwrap();
let retrieved2 = kubo_cat.stdout.clone();
let test1_pass = retrieved1 == content1;
let test2_pass = retrieved2 == content2;
let status = if test1_pass && test2_pass { TestStatus::Pass } else { TestStatus::Fail };
let evidence = format!(
"Test 1: Kubo adds, ferripfs reads\n\
Original content: {:?}\n\
CID: {}\n\
Retrieved by ferripfs: {:?}\n\
Match: {}\n\
\n\
Test 2: ferripfs adds, Kubo reads\n\
Original content: {:?}\n\
CID: {}\n\
Retrieved by Kubo: {:?}\n\
Match: {}",
String::from_utf8_lossy(content1),
cid1,
String::from_utf8_lossy(&retrieved1),
test1_pass,
String::from_utf8_lossy(content2),
cid2,
String::from_utf8_lossy(&retrieved2),
test2_pass
);
// ... record result
}| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-5.1 | Pin types: direct, recursive, indirect | boxo/pinning/pinner/pin.go |
Unit |
| REQ-5.2 | Pinner trait: IsPinned, Pin, Unpin, PinWithMode | boxo/pinning/pinner/pin.go |
Unit |
| REQ-5.3 | Recursive pin follows all links | boxo/pinning/pinner/dspinner |
Unit |
| REQ-5.4 | Direct pin only pins the root | boxo/pinning/pinner/dspinner |
Unit |
| REQ-5.5 | Indirect pins created by recursive pins | boxo/pinning/pinner/dspinner |
Unit |
| REQ-5.6 | Pin names support | boxo/pinning/pinner/dspinner |
Unit |
| REQ-5.7 | Pin persistence across restart | boxo/pinning/pinner/dspinner |
Integration |
| REQ-5.8 | ferripfs pin add <cid> pins recursively |
core/commands/pin/pin.go |
Integration |
| REQ-5.9 | ferripfs pin add --recursive=false pins directly |
core/commands/pin/pin.go |
Integration |
| REQ-5.10 | ferripfs pin add --name=<name> adds pin name |
core/commands/pin/pin.go |
Integration |
| REQ-5.11 | ferripfs pin rm <cid> removes pin |
core/commands/pin/pin.go |
Integration |
| REQ-5.12 | ferripfs pin ls lists all pins |
core/commands/pin/pin.go |
Integration |
| REQ-5.13 | ferripfs pin ls --type=recursive filters |
core/commands/pin/pin.go |
Integration |
| REQ-5.14 | ferripfs pin ls --names shows pin names |
core/commands/pin/pin.go |
Integration |
| REQ-5.15 | ferripfs pin verify verifies pin integrity |
core/commands/pin/pin.go |
Integration |
| REQ-5.16 | ferripfs pin update <old> <new> updates pin |
core/commands/pin/pin.go |
Integration |
| REQ-5.17 | GC does not remove pinned blocks | core/commands/repo.go |
Integration |
| REQ-5.18 | GC removes unpinned blocks | core/commands/repo.go |
Integration |
| REQ-5.19 | ferripfs repo gc runs garbage collection |
core/commands/repo.go |
Integration |
| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-6.1 | IPLD DAG node creation | boxo/ipld/merkledag |
Unit |
| REQ-6.2 | dag-pb codec support | go-codec-dagpb |
Unit |
| REQ-6.3 | dag-cbor codec support | go-ipld-cbor |
Unit |
| REQ-6.4 | dag-json codec support | go-ipld-prime/codec/dagjson |
Unit |
| REQ-6.5 | raw codec support | N/A | Unit |
| REQ-6.6 | ferripfs dag get <cid> retrieves node |
core/commands/dag/dag.go |
Integration |
| REQ-6.7 | ferripfs dag get --output-codec transcodes |
core/commands/dag/dag.go |
Integration |
| REQ-6.8 | ferripfs dag put stores DAG node |
core/commands/dag/dag.go |
Integration |
| REQ-6.9 | ferripfs dag put --input-codec specifies input |
core/commands/dag/dag.go |
Integration |
| REQ-6.10 | ferripfs dag put --store-codec specifies storage |
core/commands/dag/dag.go |
Integration |
| REQ-6.11 | ferripfs dag resolve <path> resolves IPLD path |
core/commands/dag/dag.go |
Integration |
| REQ-6.12 | ferripfs dag stat <cid> shows DAG statistics |
core/commands/dag/dag.go |
Integration |
| REQ-6.13 | ferripfs dag export <cid> exports CAR |
core/commands/dag/dag.go |
Integration |
| REQ-6.14 | ferripfs dag import <car> imports CAR |
core/commands/dag/dag.go |
Integration |
| REQ-6.15 | Path traversal through DAG links | boxo/path |
Unit |
| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-7.1 | libp2p host initialization | core/node/libp2p/host.go |
Unit |
| REQ-7.2 | TCP transport | core/node/libp2p/transport.go |
Integration |
| REQ-7.3 | QUIC transport | core/node/libp2p/transport.go |
Integration |
| REQ-7.4 | WebSocket transport | core/node/libp2p/transport.go |
Integration |
| REQ-7.5 | Noise encryption | core/node/libp2p/sec.go |
Unit |
| REQ-7.6 | Yamux multiplexing | core/node/libp2p/smux.go |
Unit |
| REQ-7.7 | mDNS local discovery | core/node/libp2p/discovery.go |
Integration |
| REQ-7.8 | Connection manager (limits) | core/node/libp2p/rcmgr.go |
Unit |
| REQ-7.9 | NAT traversal (UPnP) | core/node/libp2p/nat.go |
Integration |
| REQ-7.10 | AutoNAT service | core/node/libp2p/nat.go |
Integration |
| REQ-7.11 | Hole punching | core/node/libp2p/nat.go |
Integration |
| REQ-7.12 | Circuit relay client | core/node/libp2p/relay.go |
Integration |
| REQ-7.13 | Identify protocol | core/node/libp2p/host.go |
Integration |
| REQ-7.14 | ferripfs daemon starts and listens |
cmd/ipfs/kubo/daemon.go |
Integration |
| REQ-7.15 | ferripfs daemon announces addresses |
cmd/ipfs/kubo/daemon.go |
Integration |
| REQ-7.16 | ferripfs id shows peer info |
core/commands/id.go |
Integration |
| REQ-7.17 | ferripfs swarm peers lists connected peers |
core/commands/swarm.go |
Integration |
| REQ-7.18 | ferripfs swarm connect <addr> connects to peer |
core/commands/swarm.go |
Integration |
| REQ-7.19 | ferripfs swarm disconnect <addr> disconnects |
core/commands/swarm.go |
Integration |
| REQ-7.20 | ferripfs swarm addrs lists known addresses |
core/commands/swarm.go |
Integration |
| REQ-7.21 | ferripfs ping <peer> measures latency |
core/commands/ping.go |
Integration |
| REQ-7.22 | ferripfs bootstrap list shows bootstrap peers |
core/commands/bootstrap.go |
Integration |
| REQ-7.23 | ferripfs bootstrap add <addr> adds bootstrap |
core/commands/bootstrap.go |
Integration |
| REQ-7.24 | ferripfs bootstrap rm <addr> removes bootstrap |
core/commands/bootstrap.go |
Integration |
| REQ-7.25 | ferripfs shutdown stops daemon gracefully |
core/commands/shutdown.go |
Integration |
| REQ-7.26 | ferripfs can connect to Kubo peer | N/A | Integration |
| REQ-7.27 | Kubo can connect to ferripfs peer | N/A | Integration |
| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-8.1 | Kademlia DHT client mode | core/node/libp2p/routing.go |
Integration |
| REQ-8.2 | Kademlia DHT server mode | core/node/libp2p/routing.go |
Integration |
| REQ-8.3 | DHT bootstrap | go-libp2p-kad-dht |
Integration |
| REQ-8.4 | Content routing: Provide | go-libp2p-kad-dht |
Integration |
| REQ-8.5 | Content routing: FindProviders | go-libp2p-kad-dht |
Integration |
| REQ-8.6 | Peer routing: FindPeer | go-libp2p-kad-dht |
Integration |
| REQ-8.7 | Value store: PutValue (IPNS) | go-libp2p-kad-dht |
Integration |
| REQ-8.8 | Value store: GetValue (IPNS) | go-libp2p-kad-dht |
Integration |
| REQ-8.9 | ferripfs dht findprovs <cid> finds providers |
core/commands/dht.go |
Integration |
| REQ-8.10 | ferripfs dht findpeer <peerid> finds peer |
core/commands/dht.go |
Integration |
| REQ-8.11 | ferripfs dht provide <cid> announces content |
core/commands/dht.go |
Integration |
| REQ-8.12 | ferripfs routing get <key> retrieves value |
core/commands/routing.go |
Integration |
| REQ-8.13 | ferripfs routing put <key> <value> stores value |
core/commands/routing.go |
Integration |
| REQ-8.14 | ferripfs routing findprovs <cid> finds providers |
core/commands/routing.go |
Integration |
| REQ-8.15 | ferripfs routing findpeer <peerid> finds peer |
core/commands/routing.go |
Integration |
| REQ-8.16 | ferripfs stats dht shows DHT stats |
core/commands/stat_dht.go |
Integration |
| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-9.1 | Bitswap protocol /ipfs/bitswap/1.2.0 | boxo/bitswap |
Integration |
| REQ-9.2 | Bitswap wantlist management | boxo/bitswap/wantlist |
Unit |
| REQ-9.3 | Bitswap block requests | boxo/bitswap/client |
Integration |
| REQ-9.4 | Bitswap block serving | boxo/bitswap/server |
Integration |
| REQ-9.5 | Bitswap ledger tracking | boxo/bitswap/decision |
Unit |
| REQ-9.6 | ferripfs bitswap stat shows stats |
core/commands/bitswap.go |
Integration |
| REQ-9.7 | ferripfs bitswap wantlist shows wanted blocks |
core/commands/bitswap.go |
Integration |
| REQ-9.8 | ferripfs bitswap ledger <peer> shows ledger |
core/commands/bitswap.go |
Integration |
| REQ-9.9 | Content exchange: ferripfs fetches from Kubo | N/A | Integration |
| REQ-9.10 | Content exchange: Kubo fetches from ferripfs | N/A | Integration |
| REQ-9.11 | Large file transfer (100MB+) | N/A | Integration |
| REQ-9.12 | Concurrent block requests | boxo/bitswap |
Integration |
#[test]
fn req_9_9_10_cross_impl_content_exchange() {
let req = Requirement {
id: "REQ-9.9/10".into(),
title: "Content exchange between ferripfs and Kubo over network".into(),
description: "Nodes must be able to fetch content from each other over the network".into(),
kubo_source: "N/A (interoperability requirement)".into(),
verification_steps: vec![
"Start Kubo daemon on port 4001".into(),
"Start ferripfs daemon on port 4002".into(),
"Add unique content to Kubo".into(),
"Add unique content to ferripfs".into(),
"Connect nodes via swarm connect".into(),
"Fetch Kubo content from ferripfs using CID".into(),
"Fetch ferripfs content from Kubo using CID".into(),
"Verify fetched content matches original".into(),
],
};
// Start Kubo daemon
let kubo_repo = tempdir().unwrap();
init_kubo_repo(&kubo_repo);
let kubo_daemon = Command::new("ipfs")
.env("IPFS_PATH", kubo_repo.path())
.args(["daemon", "--offline=false"])
.spawn()
.unwrap();
wait_for_daemon(&kubo_repo, 4001);
// Start ferripfs daemon
let ferripfs_repo = tempdir().unwrap();
init_ferripfs_repo(&ferripfs_repo);
let ferripfs_daemon = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", ferripfs_repo.path())
.args(["daemon"])
.spawn()
.unwrap();
wait_for_daemon(&ferripfs_repo, 4002);
// Add content to each
let kubo_content = b"Content from Kubo node for cross-exchange test";
let kubo_cid = add_content_to_kubo(&kubo_repo, kubo_content);
let ferripfs_content = b"Content from ferripfs node for cross-exchange test";
let ferripfs_cid = add_content_to_ferripfs(&ferripfs_repo, ferripfs_content);
// Get peer addresses and connect
let kubo_addr = get_peer_addr(&kubo_repo, "kubo");
let ferripfs_addr = get_peer_addr(&ferripfs_repo, "ferripfs");
connect_peers(&ferripfs_repo, &kubo_addr);
// Cross-fetch
let fetched_from_kubo = Command::new("./target/debug/ferripfs")
.env("IPFS_PATH", ferripfs_repo.path())
.args(["cat", &kubo_cid])
.output()
.unwrap();
let fetched_from_ferripfs = Command::new("ipfs")
.env("IPFS_PATH", kubo_repo.path())
.args(["cat", &ferripfs_cid])
.output()
.unwrap();
// Cleanup
kubo_daemon.kill();
ferripfs_daemon.kill();
let kubo_fetch_ok = fetched_from_kubo.stdout == kubo_content;
let ferripfs_fetch_ok = fetched_from_ferripfs.stdout == ferripfs_content;
let status = if kubo_fetch_ok && ferripfs_fetch_ok {
TestStatus::Pass
} else {
TestStatus::Fail
};
let evidence = format!(
"Network Content Exchange Test\n\
==============================\n\
\n\
Kubo node:\n\
- Address: {}\n\
- Added content: {:?}\n\
- CID: {}\n\
\n\
ferripfs node:\n\
- Address: {}\n\
- Added content: {:?}\n\
- CID: {}\n\
\n\
Cross-fetch results:\n\
- ferripfs fetched from Kubo: {} bytes, match: {}\n\
- Kubo fetched from ferripfs: {} bytes, match: {}\n\
\n\
Overall: {}",
kubo_addr,
String::from_utf8_lossy(kubo_content),
kubo_cid,
ferripfs_addr,
String::from_utf8_lossy(ferripfs_content),
ferripfs_cid,
fetched_from_kubo.stdout.len(),
kubo_fetch_ok,
fetched_from_ferripfs.stdout.len(),
ferripfs_fetch_ok,
if kubo_fetch_ok && ferripfs_fetch_ok { "SUCCESS" } else { "FAILURE" }
);
// ... record result
}| ID | Requirement | Kubo Source | Test Type |
|---|---|---|---|
| REQ-10.1 | IPNS record protobuf format | boxo/ipns/pb |
Unit |
| REQ-10.2 | IPNS record signing (Ed25519) | boxo/ipns |
Unit |
| REQ-10.3 | IPNS record validation | boxo/ipns |
Unit |
| REQ-10.4 | IPNS record versioning (sequence) | boxo/ipns |
Unit |
| REQ-10.5 | IPNS publish to DHT | boxo/namesys/publisher.go |
Integration |
| REQ-10.6 | IPNS resolve from DHT | boxo/namesys/resolver.go |
Integration |
| REQ-10.7 | IPNS republisher service | boxo/namesys/republisher |
Integration |
| REQ-10.8 | Key generation: Ed25519 (default) | core/commands/keystore.go |
Integration |
| REQ-10.9 | Key generation: RSA | core/commands/keystore.go |
Integration |
| REQ-10.10 | Key generation: ECDSA | core/commands/keystore.go |
Integration |
| REQ-10.11 | Key generation: Secp256k1 | core/commands/keystore.go |
Integration |
| REQ-10.12 | ferripfs key gen <name> generates key |
core/commands/keystore.go |
Integration |
| REQ-10.13 | ferripfs key list lists keys |
core/commands/keystore.go |
Integration |
| REQ-10.14 | ferripfs key rename <old> <new> renames |
core/commands/keystore.go |
Integration |
| REQ-10.15 | ferripfs key rm <name> removes key |
core/commands/keystore.go |
Integration |
| REQ-10.16 | ferripfs key export <name> exports key |
core/commands/keystore.go |
Integration |
| REQ-10.17 | ferripfs key import <name> <file> imports key |
core/commands/keystore.go |
Integration |
| REQ-10.18 | ferripfs name publish <cid> publishes IPNS |
core/commands/name/publish.go |
Integration |
| REQ-10.19 | ferripfs name publish --key=<name> uses key |
core/commands/name/publish.go |
Integration |
| REQ-10.20 | ferripfs name resolve <name> resolves IPNS |
core/commands/name/resolve.go |
Integration |
| REQ-10.21 | ferripfs name inspect <record> inspects |
core/commands/name/inspect.go |
Integration |
| REQ-10.22 | IPNS record published by ferripfs resolved by Kubo | N/A | Integration |
| REQ-10.23 | IPNS record published by Kubo resolved by ferripfs | N/A | Integration |
- 20 requirements covering files subcommands
- REQ-11.1 through REQ-11.20
- 10 requirements covering GossipSub/FloodSub
- REQ-12.1 through REQ-12.10
- 15 requirements covering path/subdomain gateway
- REQ-13.1 through REQ-13.15
- 10 requirements covering JSON-RPC parity
- REQ-14.1 through REQ-14.10
- All remaining CLI commands
- Full sharness test suite porting
- Performance benchmarking
- Documentation completion
# .github/workflows/phase-N.yml
name: Phase N Compliance
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Kubo (for interop tests)
run: |
wget https://dist.ipfs.tech/kubo/v0.39.0/kubo_v0.39.0_linux-amd64.tar.gz
tar xzf kubo_v0.39.0_linux-amd64.tar.gz
sudo mv kubo/ipfs /usr/local/bin/
- name: Build
run: cargo build --workspace
- name: Run Phase N Tests
run: cargo test --test phase_n_compliance -- --nocapture
- name: Check Coverage
run: |
cargo tarpaulin --out Json --output-dir coverage
python3 scripts/check_coverage.py coverage/tarpaulin-report.json 85
- name: Generate Compliance Report
run: cargo test --test phase_n_compliance -- --nocapture > compliance_report.txt
- name: Upload Compliance Report
uses: actions/upload-artifact@v4
with:
name: phase-n-compliance-report
path: compliance_report.txtThis plan provides:
-
Granular requirements - Each phase has specific, testable requirements with IDs
-
85% coverage target - Enforced per phase via cargo-tarpaulin
-
Human-readable compliance reports - Every integration test produces detailed output showing:
- Requirement ID and title
- Description of what's being tested
- Reference to Kubo source
- Step-by-step verification instructions
- Evidence of compliance
- Pass/fail status
-
Kubo interoperability - Critical requirements explicitly test cross-implementation compatibility
-
Traceability - Every requirement links to Kubo source code
The compliance report format ensures any human reviewer can:
- Understand what each requirement specifies
- See exactly what test validates it
- Verify the evidence independently
- Confirm the requirement is met