Skip to content

Tsehla/ResiFS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ResiFS — Resilient, Self-Linking Chunk Storage System

credits

Thanks chatgpt for creating this documentation and example script (which i didnt test, this is were u the reader comes in)
Thanks open source community - This is my way of giving back to current en future comunity
Thanks crackers and hackers for allowing the younger poor me asses to things i would not have afforded not matter what i did. In my heart i see myself as one of you ( only that due to certain events i had to take a different path from technology to hacking en cracking spirituality ) with the same goal of opening the mysterious to overyone.


“Outlive us” — a decentralized, long-lasting way to store, share, and preserve digital data.


🧭 Overview

ResiFS is a decentralized file distribution protocol that breaks files into self-linking, optionally encrypted chunks. Each chunk contains metadata about its neighbors, making the file reconstructable without a central server or seeder. The goal is to build a storage system that survives seed loss, takedowns, and central coordination failures — while remaining lightweight, user-controllable, and extensible.

ResiFS provides:

  • Long-term survivability even with minimal hosting
  • Human-readable chunk names with obfuscation
  • Hybrid public/private sharing with partial encryption
  • Optional streaming and partial recovery
  • Support for anonymous, open distribution and private control

🚀 Goals

  • Remove reliance on seeders
  • Survive content takedowns and censorship
  • Empower anyone to mirror and distribute anonymously
  • Be simple enough to implement or adopt using existing infrastructure
  • Outlast us

📦 How It Works: Chunking, Linking & Discovery

Chunk Anatomy

Each file is split into multiple chunks, and each chunk contains:

Field Description
chunk_name Human-readable, unique filename (e.g. blue-moon-abc13.jpg)
previous_chunk Filename of the previous chunk (with extension)
next_chunk Filename of the next chunk (with extension)
data Actual chunk content (encrypted or plain)
decrypt_key_piece Optional: portion of the decryption key
chunk_extension File extension for obfuscation (real type declared in metadata)
user_text Optional: embedded text or notes (e.g., for hidden messages, metadata, etc.)

Self-Linking Explained

Each chunk points backward and forward, forming a chain (or DAG) structure that can be:

  • Navigated without a tracker or index
  • Traversed from any chunk (not just the start)
  • Verified using hashes or chunk names

Example chunk chain:

sunset-dawn-1f91a.jpg → forest-rain-402bd.tif → temple-fire-9ab33.mkv

Each chunk embeds the exact name of its neighbors, enabling decentralized traversal and retrieval.


Chunk Naming Strategy

  • Filenames combine human-readable phrases with a short hash or UID (e.g., golden-leaf-a1f3b.png)
  • Randomized file extensions (.jpg, .txt, .bin, etc.) obscure purpose
  • Full filename is used in linkage — not just the UID — adding anti-pattern detection value

This helps the system hide in plain sight while being readable and organized.


🔐 Optional Encryption & Hybrid Key Chunk System

ResiFS supports flexible encryption mechanisms for security and control:

🔸 Hybrid Key Chunks

You may choose to:

  • Split the file decryption key across a few specific key chunks
  • Keep key chunks private or hosted elsewhere
  • Distribute other chunks publicly

This way, even if someone finds all data chunks, they can’t decrypt without the key chunks.

Bonus: The system still works — chunk traversal and validation are possible without key chunks.

Use Case Example

  • You keep chunks 1, 2, and 3 (key chunks) on a secure private server.
  • You publicly mirror chunks 4–99.
  • Without 1–3, others can’t reconstruct the file.

🛠️ Optional Features & Configuration

Feature Purpose
✅ Self-linking chunks Enables decentralized reassembly
✅ Randomized chunk extensions Obfuscates content type, evades MIME-based filtering
✅ Human-readable + UID names Increases usability and uniqueness
✅ Key chunk isolation Enables secure access gating
✅ Embedded text fields Hidden messages, authorship, indexing, etc.
✅ Variable chunk sizes Prevents pattern recognition, reduces deduplication matching
✅ Multi-version file graphs Permits multiple chunking paths for same file to resist targeting
✅ Optional manifests Allow accelerated discovery (but not required)

📂 File Discovery and Use

You can retrieve a file by:

  1. Searching across public storage for known chunk names (via DHT, HTTP mirror, or search engine indexing)
  2. Reading the next_chunk and previous_chunk fields inside each chunk
  3. Recursively building the full chunk graph
  4. Assembling the chunks in order and applying the decryption key if needed

Manifest Files (Optional)

Anyone (even downstream users) can build a manifest file containing:

  • List of all chunk filenames and their storage URLs
  • Metadata like hash validation or total file size
  • Decryption key fragments (if available)

Manifests are optional but can greatly speed up retrieval.


📈 Comparison With Existing Systems

Feature BitTorrent IPFS ResiFS (this)
Seeder-free survival ⚠️ Partial ✅ Fully supported
Self-linking chunks ✅ Yes
Decentralized reassembly ⚠️ CID only ✅ Chunk-driven
Hybrid private/public split ⚠️ Via VPN ✅ Native
Encryption model External Optional ✅ Chunk-native
Multiple file versions ⚠️ Manual ✅ Native support

🧬 Technical Design Philosophy

ResiFS emphasizes:

  • Resilience: The file must be recoverable even if the uploader disappears
  • Obfuscation: No obvious patterns that attract takedown attempts
  • Modularity: Users can choose what features they want (encryption, manifests, etc.)
  • Simplicity: Built with existing storage and retrieval mechanisms (HTTP, S3, FTP, torrents)
  • Accessibility: Anyone can host, share, mirror — no need for special nodes

✊ Motivation

ResiFS was born out of a desire to preserve data in ways that don’t require constant human intervention or trust in platforms. Raised in a township in South Africa, I grew up fascinated by tech, helped along by hackers, crackers, and communities who gave everything away for free — sometimes at risk to themselves.

This project is my return gift — a tribute to those people who built and gave freely. Some are gone, some are jailed, some still risk their freedom.
ResiFS is for them. It’s for all of us.


🛠 Repository Layout

resifs/
├── README.md # This file
├── docs/ # Optional technical documentation
│ ├── protocol.md # Detailed spec and protocol behavior
│ ├── formats.md # Example chunk and manifest formats
├── examples/ # Sample chunks, names, encrypted payloads
├── tools/ # Utilities for chunking, decrypting, manifesting
│ ├── chunker.py
│ ├── assembler.py
│ └── discoverer.py
├── LICENSE


🔄 Getting Started

  1. Chunk a file using a tool that generates:
    • Unique, human-readable filenames with random extensions
    • Embedded previous/next chunk filenames
    • Optional key chunks or encryption
  2. Upload chunks to public or mixed mirrors (Pastebin, Dropbox, FTP, etc.)
  3. Optionally generate a manifest (can be public or private)
  4. Distribute only chunk names or a starting manifest URL
  5. Anyone can reconstruct the file — if they find the chunks
  6. This all should idealy be handled by an aplication or scripts automatically

🤝 Contributions & Improvements

This is a community-first protocol. All ideas, pull requests, critiques, and forks are welcome.
Help build tools, write clients, design naming schemes, and build resilient networks.

Let’s outlive the cloud.


📝 License

Open-source and free forever. You can reuse, rebrand, repurpose — just don’t lock it away.


Built for those who preserve, in spite of all odds.

About

A self-replicating file storage protocol built on distributed chunks that know how to find each other. No seeders, no central servers — just pure resilience. Designed for simplicity, privacy, and long-term access.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages