InterPlanetary File System (IPFS)

• InterPlanetary File System (IPFS) is a decentralized, content-addressed storage system that organizes data as immutable objects using a Merkle Directed Acyclic Graph.
• It integrates protocols from BitTorrent, Git, and Kademlia DHTs to enable efficient peer discovery, deduplication, and secure, trustless data exchanges.
• IPFS underpins diverse applications such as permanent web hosting, distributed databases, and backup systems, ensuring data integrity with cryptographic enforcement.

The InterPlanetary File System (IPFS) is a peer-to-peer, content-addressed distributed file system that aims to unify and generalize file storage and sharing across all computing devices. IPFS combines principles and mechanisms from BitTorrent, Git, and Kademlia DHTs to yield a decentralized, high-throughput, versioned storage network, supporting robust integrity guarantees, flexible linking, efficient deduplication, and self-certifying naming. By organizing all data as immutable objects in a Merkle Directed Acyclic Graph (DAG), and providing a set of layered, interoperable protocols, IPFS serves as an infrastructure for versioned file systems, permanent web applications, distributed databases, and censorship-resistant content distribution. Nodes in IPFS do not require mutual trust and the system is designed with no single point of failure (Benet, 2014).

1. Architectural and Protocol Foundations

IPFS architecture is modular, comprised of the following main subsystems:

• Identities: Each node generates a cryptographic public-private key pair, with its NodeId defined as the hash of its public key, establishing a self-certifying identity analogous to S/Kademlia’s approach.
• Networking: Nodes communicate over various transport protocols (WebRTC DataChannels, SCTP, uTP/LEDBAT), leveraging multiaddr addressing for extensibility with NAT traversal (ICE), reliability, and authenticity.
• Routing: Content and node discovery employ a distributed hash table (DHT) based on Kademlia variants (Coral, S/Kademlia), facilitating peer lookup by NodeId and routing of provider records for content.
• Block Exchange: BitSwap, inspired but generalized from BitTorrent, orchestrates persistent, incentive-aligned exchange of content-indexed blocks across swarms.
• Storage Objects: Data is decomposed into content-addressed (multihash-indexed) blocks, forming an extensible Merkle DAG that maps hierarchical file/directory (and versioning) structures and supports a variety of upper-layer data models.
• Versioning: The immutable DAG enables Git-like versioned file systems, where each commit is a snapshot referencing parent(s); any mutations create new objects and links.
• Self-certifying Naming: Mutable pointers are realized via IPNS, which binds “/ipns/NodeId” to a current root CID, authenticated by signatures over updates (Benet, 2014).

This layered stack forms a highly fault-tolerant and composable substrate for durable, distributed, and trustless content systems.

2. Cryptographically Enforced Content Addressing

IPFS uniquely identifies all content by the hash (digest) of its data and metadata (links). The addressing scheme uses a self-describing “multihash” format:

$$\text{Multihash} = \langle \text{function code}, \text{digest length}, \text{digest bytes} \rangle$$

This approach enforces:

• Integrity: Any bit-change in content yields a different hash, ensuring object tamper resistance.
• Deduplication: Identical objects are by definition address-equal, naturally deduplicating storage.
• Security/Extensibility: Support for migrating to new cryptographic hash functions by embedding the function code.

Notably, all linkages within the object graph reference children by their content hash, making the global object store self-rooted; trust and verification are cryptographically enforced rather than dependent on location or authority (Benet, 2014).

3. Data Structures: The Merkle Directed Acyclic Graph

All data in IPFS is organized as a Merkle DAG, a generalization of typical Merkle trees:

• Nodes (IPFS Objects):
  • Arbitrary data payload
  • Set of content-addressed links (multihashes of child objects)
• Composability: DAGs can encode classic file trees, blocklists (split files), version snapshots (via commit objects), and can extend to blockchains and other application-specific graphs.

This structure underpins:

• Auditability and Integrity: Hash-linked structure makes tamper-detection and object graph verification trivial.
• Versioning: Branches and histories are natively supported as in Git, with graph navigation for snapshots, diffs, or rollbacks.
• Universality: The system subsumes blockchains, key-value stores, and generic databases through DAG composition (Benet, 2014).

4. Overlay Routing, Exchange, and Incentives

4.1 Distributed Hash Table (DHT)

• Peer and Data Discovery: Nodes use a Kademlia-based DHT, efficiently locating peers (log₂(n) hops) and mapping content identifiers to serving peers.
• Provider Records: For large objects, the DHT stores provider pointers (NodeIds), not block data; “small” (<1KB) values may be stored directly.
• Mutable State: IPNS and publish–subscribe channels utilize the DHT for dynamic name resolution and update registration (Benet, 2014).

4.2 BitSwap Block Exchange

• Generalized Trading: Peers maintain a want_list and have_list, negotiating block exchanges over a persistent credit-based market.
• Incentive Ledger: Each peer-pair records bytes sent/received. Probability of serving further blocks is computed as:

$$r = \frac{\text{bytes sent}}{\text{bytes received} + 1} \qquad P(\mathrm{send} | r) = 1 - \frac{1}{1 + \exp(6-3r)}$$

• Anti-leeching: The system probabilistically throttles freeloaders and rewards reciprocation, mitigating resource abuse (Benet, 2014).

5. Self-Certifying, Mutable Naming (IPNS)

To support pointer mutability atop an immutable DAG, IPFS incorporates the InterPlanetary Naming System (IPNS):

• Namespace per Node: Each /ipns/NodeId acts as a root for mutable references, with the NodeId as hash(public_key).
• Authenticated Updates: Name updates are signed; validation uses the embedded public key.
• Usability Enhancements: Human-friendly resolution with DNS TXT records, proquint encoding, and peer links.

This mechanism reconciles the need for updateability within a permanent, content-addressed substrate, maintaining end-to-end cryptographic trust (Benet, 2014).

6. Representative Use Cases and Applications

Key applications enabled by IPFS’s architecture include:

• Permanent Web: Immutable references and local pinning ensure long-term retrievability, combating “link rot.”
• Filesystem Mounting: FUSE or kernel modules can offer /ipfs/ (static content) and /ipns/ (mutable pointers) mount points.
• Package Distribution: Versioned, deduplicated package repositories.
• Backup and Synchronization: Automated, versioned, distributed backups.
• Content Distribution Networks (CDNs): Decentralized large object distribution with built-in integrity checking.
• General Databases/Blockchains: Merkle DAG extensibility supports data models for blockchains and distributed key-value stores.
• Encrypted Sharing: Object-level cryptography enables secure and selective sharing (Benet, 2014).

7. Security, Robustness, and Systemic Properties

• Decentralization: No single point of control; node reliability and data persistence are ensured by distributed pinning and multi-party participation.
• Trustless Operation: Nodes do not trust one another; all verification is cryptographic.
• Resilience to Attack: The modular incentive and namespace design, coupled with content addressing, make massive data censorship, tampering, or coordinated loss infeasible under typical adversary assumptions.

IPFS is paradigmatic of a resilient, scalable, and composable distributed data infrastructure, merging decades of P2P, hash-linked storage, and self-certifying systems into a practical framework for the permanent, decentralized web (Benet, 2014).