Decentralised file storage

Decentralised file storage denotes a class of infrastructure protocols and networks that disaggregate object/blob persistence across geographically distributed, mutually untrusting node operators using content-addressing, cryptographic accountability, and (in most implementations) crypto-economic…

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:hasPart infra:ContentIdentifier))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:hasPart infra:StorageProvider))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:hasPart infra:RetrievalMarket))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:hasPart infra:PinningService))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:hasPart infra:ErasureCoding))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:hasPart infra:DistributedHashTable))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:hasPart infra:StorageProof))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:hasPart infra:GatewayService))

## Dependency Relationships
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:requires infra:LibP2P))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:requires infra:CryptographicHashFunction))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:requires infra:PublicKeyInfrastructure))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:requires infra:NetworkBandwidth))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:requires infra:DiskStorage))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:dependsOn infra:Blockchain))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:dependsOn infra:PublicKeyCryptography))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:dependsOn infra:ZkSNARK))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:dependsOn infra:MerkleTree))

## Capability Relationships
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:enables infra:CensorshipResistance))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:enables infra:NFTMetadataPersistence))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:enables infra:VerifiableStorage))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:enables infra:Permaweb))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:enables infra:DataSovereignty))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:supports infra:DecentralizedApplication))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:supports infra:AIDatasetDistribution))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:supports infra:BlockchainDataArchival))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:supports infra:VideoStreaming))

## Implementation Relationships
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:implements infra:ProofOfReplication))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:implements infra:ProofOfSpacetime))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:implements infra:BitswapProtocol))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:implements infra:KademliaDHT))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:implements infra:ReedSolomonErasureCoding))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:uses infra:CID))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:uses infra:IPLD))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:uses infra:CARFile))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:uses infra:UCAN))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:uses infra:Multihash))

## Reduction Relationships
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:reduces infra:SinglePointOfFailure))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:reduces infra:VendorLockIn))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:reduces infra:CensorshipVulnerability))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:reduces infra:OperatorTrustRequirement))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:reduces infra:DataLossRisk))

## Association Relationships
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:relatedTo infra:DistributedLedgerTechnology))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:relatedTo infra:SmartContract))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:relatedTo infra:Tokenomics))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:contrastsWith infra:CloudObjectStorage))
SubClassOf(infra:DecentralisedFileStorage
  ObjectSomeValuesFrom(infra:contrastsWith infra:ContentDeliveryNetwork))

## Data Properties (Characteristics)
DataPropertyAssertion(infra:hasIdentifier infra:DecentralisedFileStorage "IF-1042"^^xsd:string)
DataPropertyAssertion(infra:authorityScore infra:DecentralisedFileStorage "0.87"^^xsd:decimal)
DataPropertyAssertion(infra:filecoinCommittedCapacityEiB infra:DecentralisedFileStorage "21"^^xsd:integer)
DataPropertyAssertion(infra:arweavePermawebPB infra:DecentralisedFileStorage "150"^^xsd:integer)
DataPropertyAssertion(infra:storjNodeOperators infra:DecentralisedFileStorage "24000"^^xsd:integer)
DataPropertyAssertion(infra:ipfsPublicNodes infra:DecentralisedFileStorage "250000"^^xsd:integer)
DataPropertyAssertion(infra:filecoinStorageProviders infra:DecentralisedFileStorage "3800"^^xsd:integer)

## Property Constraints
SubClassOf(infra:DecentralisedFileStorage
  DataAllValuesFrom(infra:isContentAddressed xsd:boolean))
SubClassOf(infra:DecentralisedFileStorage
  DataMinCardinality(1 infra:hasReplicationFactor xsd:integer))
SubClassOf(infra:DecentralisedFileStorage
  DataSomeValuesFrom(infra:hasIncentiveMechanism xsd:string))
SubClassOf(infra:DecentralisedFileStorage
  DataMaxCardinality(1 infra:hasNativeToken xsd:string))

## Annotations
AnnotationAssertion(rdfs:label infra:DecentralisedFileStorage "Decentralised File Storage"@en)
AnnotationAssertion(rdfs:comment infra:DecentralisedFileStorage "Class of infrastructure protocols disaggregating object storage across untrusting operators via content-addressing and crypto-economic incentives, spanning content-addressed peer networks (IPFS), permanent storage chains (Arweave), incentivized retrieval markets (Filecoin with FVM 2023), erasure-coded distributed clouds (Storj), proof-of-storage chains (Sia), Ethereum-companion (Swarm), Sui-native (Walrus 2025), Solana-native (Shadow Drive), enabling censorship-resistant publishing, NFT metadata persistence, AI dataset distribution; contrasted with cloud object storage (S3/GCS/R2) and CDNs (Cloudflare/Akamai)."@en)
AnnotationAssertion(dcterms:identifier infra:DecentralisedFileStorage "IF-1042"^^xsd:string)
AnnotationAssertion(dcterms:subject infra:DecentralisedFileStorage "Web3 Infrastructure, Content-Addressed Storage, Cryptoeconomic Protocols, Distributed Systems"@en)

)

Property Characteristics

AsymmetricObjectProperty(infra:requires) AsymmetricObjectProperty(infra:enables) AsymmetricObjectProperty(infra:implements) AsymmetricObjectProperty(infra:reduces) TransitiveObjectProperty(infra:dependsOn) FunctionalDataProperty(infra:hasReplicationFactor) FunctionalDataProperty(infra:authorityScore)

About Decentralised File Storage

• Decentralised file storage is the infrastructure category covering protocols and networks that persist arbitrary binary objects (files, blobs, chunks) across many independently operated nodes without trusting any single operator, replacing the operator-trust model of cloud object storage (where AWS, Google, Microsoft, or Cloudflare physically holds and could lose/serve/censor data) with cryptographic verifiability (content-addressed hashes, storage proofs) and, in most cases, crypto-economic incentives (native tokens paying storage providers, slashing collateral on proven faults). The category emerged from the post-Snowden, post-DAO-hack period (2015-2018) as Ethereum, Bitcoin, and the broader web3 thesis collided with Juan Benet’s content-addressing thesis articulated in the original IPFS paper (Benet 2014, “IPFS - Content Addressed, Versioned, P2P File System”). Today the landscape comprises a dozen viable production networks holding aggregate exabytes of data, embedded in NFT markets, DeFi front-ends, AI dataset commons, and government open-data archives.
• The defining architectural inversion versus cloud storage is content-addressing: instead of referring to a file by where it lives (https://my-bucket.s3.amazonaws.com/cat.jpg), the system refers to it by what it is (ipfs://bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi). The Content Identifier (CID) is a self-describing multihash (typically SHA-256 over the encoded DAG) that any node can recompute from the bytes themselves. Retrieval becomes a routing problem: “who has the bytes that hash to this CID?” rather than a permission problem. This eliminates entire classes of failure (DNS hijacking, BGP attacks, single-operator key compromise, region outages) at the cost of new ones (pinning churn, retrieval latency, gateway dependencies).
• The second axis is incentive layer. Pure IPFS has no incentive: any node may garbage-collect data after 7 days (default cache TTL), so persistence requires either self-hosting a node, a commercial pinning service (Pinata, Fleek, Storacha), or a Filecoin storage deal. Filecoin solves this by adding a market: storage providers post collateral, sign deals committing to store specific CIDs for fixed terms (180 days typical, up to 540), and submit zk-SNARK proofs every ~24 hours that they still hold a unique encoding of the data (PoRep/PoSt). If they fail proofs, collateral is slashed. Arweave takes a fundamentally different approach: pay once up-front (an endowment), and the network commits to 200+ year persistence funded by the assumption that disk costs decline ~30%/year while the endowment yields enough to outpace storage costs forever (the “blockweave” + SPoRA mechanism rewards miners proportional to old-data access, incentivizing complete history retention).
• The third axis is hot vs cold storage. Hot storage (sub-second retrieval, browser-friendly, suitable for web hosting and NFT metadata) is served by IPFS gateways, Arweave gateways (arweave.net, ar.io), Storj’s S3-compatible edge, Walrus aggregators. Cold storage (minutes-to-hours retrieval, suited to archival) lives in raw Filecoin sealed sectors (must be “unsealed” before serving), Arweave’s distant historical blocks, Storj’s deeply-erasure-coded chunks. Bridging the two is the rise of decentralised CDNs: Saturn (Filecoin’s L1 retrieval network), Fleek Edge, AIOZ DCDN, Theta EdgeCacher — content-addressed but with caching/replication tuned for sub-100ms latency competitive with Cloudflare.

Architectural Primitives Common Across Networks

All credible decentralised file storage networks share a common technical substrate, the result of ~10 years of converging engineering:

• Content Identifiers (CIDs): Self-describing identifiers encoding the multihash algorithm, codec, and digest bytes. CIDv1 (baf...) is the modern form; CIDv0 (Qm...) is IPFS-legacy SHA-256-Base58. Every network either uses CIDs directly (IPFS, Filecoin, Storacha) or implements equivalents (Arweave transaction IDs, Walrus blob IDs, Swarm chunk addresses).
• DAG-PB / DAG-CBOR: Encoding formats for chunked file trees. Files larger than ~256KB are split into chunks linked by a merkle-DAG, yielding deduplication (identical chunks share storage) and partial retrieval (fetch only needed leaves).
• libp2p: The transport stack invented for IPFS now used by Filecoin, Swarm, Ethereum 2 consensus layer, Polkadot, Algorand, Storj signalling. Multiple transports (TCP, QUIC, WebTransport, WebRTC), encryption (NOISE, TLS 1.3), multiplexing (yamux, mplex), peer discovery (mDNS, bootstrap, DHT).
• Kademlia DHT: O(log n) key-based content/peer routing. IPFS public DHT contains ~250K nodes; queries traverse 20-30 hops to find content providers in 200ms-30s depending on connectivity.
• Erasure Coding: Reed-Solomon (Storj 29/80 — any 29 of 80 chunks reconstructs), tornado codes (Filecoin retrieval), RedStuff 2D coding (Walrus achieving ~5x storage overhead vs naive 25x replication). Reduces durability cost dramatically while maintaining 11+ nines durability.
• Cryptographic Storage Proofs: Filecoin’s PoRep (one-time sector seal proving unique replica via SDR Stacked Depth Robust hashing taking 12-30 hours per 32GiB sector) and PoSt (24h-interval continuing proof via Window-PoSt with rotating challenges), Arweave’s SPoRA (random access to historical blocks integrated into Hashcash-style PoW), Swarm’s redundant chunks with retrievability claims, Walrus’s KZG-style RedStuff commitments.
• Gossip and Pub/Sub: GossipSub (libp2p) provides bandwidth-efficient broadcast for storage market events (Filecoin deal announcements, IPNS record propagation, Bee postage events). Mesh-based, ~6-12 peer mesh per topic, sub-second propagation across global swarms.
• Provider Records / Indexing: Beyond DHT, modern stacks add second-tier indexers — Filecoin’s IPNI (InterPlanetary Network Indexer / cid.contact), Storacha’s indexing layer, and Hydra Boosters (super-nodes amplifying DHT throughput). Reduces tail-latency CID resolution from 30s to sub-second.
• Naming: IPNS (InterPlanetary Name System, mutable pointers signed by libp2p keys, originally IPNS-over-DHT, modern IPNS-over-PubSub), DNSLink (CNAME records pointing DNS hostnames at CIDs), ENS resolution to ipfs:// / ar:// content hashes, Unstoppable Domains, all glue decentralised storage to human-readable identifiers.

Economic Models and Incentive Design

Distinct incentive models partition the design space; understanding them is essential to evaluating any deployment decision.

• Pay-once, store-forever (Arweave): Up-front payment funds an endowment; future storage costs paid from endowment yield. Assumes long-term storage cost decline > endowment depreciation. Risk: assumption failure over century horizons; mitigated by conservative pricing (~200 year coverage at pessimistic 0.5% annual decline).
• Continuous deal markets (Filecoin): Fixed-term storage deals (typically 180-540 days) with slashable collateral. Storage providers compete on price; clients accept best bid. Renewal required for persistence. Filecoin Plus (FIL+) DataCap programme subsidises socially valuable deals with 10x deal-weight multiplier — distorts pure-market dynamics in favour of public-good data.
• Token-collateralised continuous service (Storj, Crust, Shadow Drive): Operators stake/post collateral, earn token emissions or service fees for serving valid retrievals. Closest to traditional cloud cost models, predictable monthly pricing.
• Postage-stamp prepayment (Swarm): Chunks must carry valid stamps to be served. Stamps purchased in BZZ batches; refill required to extend storage. Hybrid pay-once / continuous model.
• Stake-and-slash with deletion (Walrus): WAL token staking, storage nodes serve blobs while paid; deletion allowed via on-chain Sui object lifecycle. Aligns with GDPR-erasure requirements unlike Arweave’s immutability.
• Free + commercial pinning (IPFS layer): IPFS itself has no native incentive; persistence sold by commercial pinning services (Pinata, Storacha, Fleek, Filebase) at flat monthly rates ($5-$1000) or by Filecoin deals brokered via these services.

Threat Models and Security Properties

Decentralised file storage networks defend against distinct threat sets compared to cloud object storage:

• Operator censorship resistance: No single operator can refuse to serve content matching specific CIDs (assuming honest-majority of providers). Counter-cases: gateway-level filtering (Cloudflare gateway blocks malware lists), legal pressure on identifiable operators.
• Tamper-evidence: Content-addressing makes silent mutation impossible — any change yields a different CID. Detection is automatic; resolution (key rotation, content migration) is operational.
• Sybil resistance for storage proofs: Filecoin’s PoRep specifically requires unique encoding per sector — a single physical disk cannot satisfy multiple proofs for the same data, preventing “store-once, claim-many” Sybil attacks. Arweave’s SPoRA similarly ties block production to verifiable historical disk reads.
• Long-term durability: 11+ nines achievable via 29/80 erasure coding (Storj), 30+ replication (early Filecoin deals), Arweave’s miner economic incentive for complete history.
• Vulnerabilities: Eclipse attacks (isolating a node from the honest network), DHT poisoning (advertising false providers), proof-of-replication shortcut attacks (active research), gateway compromise (centralising trust at the HTTP boundary), key loss (UCAN/DID private keys = total loss of write access).
• Privacy: Most networks expose CIDs publicly; content privacy requires client-side encryption (Storj default, Akord on Arweave, manual on IPFS/Filecoin). Metadata leakage (file sizes, request patterns, peer identifiers) is harder to hide without onion-routing overlays.

Major Networks: Detailed Architecture

Six networks dominate production deployment in 2025-2026. Each makes distinct trade-offs along the persistence/latency/cost/decentralization quadrilateral.

1. IPFS (InterPlanetary File System)

Launched: 2015 (Protocol Labs, Juan Benet). The protocol that defined the category. Architecture: Content-addressed peer-to-peer network. Files chunked, hashed (SHA-256 default), wrapped in DAG-PB nodes, addressed by CID. Three subsystems: Bitswap (block exchange protocol — peers send WANT-HAVE / WANT-BLOCK messages, ledger-based to discourage freeloading), DHT (Kademlia content routing — providers periodically announce CIDs, retrievers walk DHT to find providers), libp2p (transport). Implementations: Kubo (Go reference, formerly go-ipfs, ~95% of public-network nodes), Helia (modern JS rewrite, replaced js-ipfs 2024, browser-native), rust-ipfs (Parity), iroh (n0 company 2023-2024, Rust, optimized for content-addressed sync). Kubo v0.30 (2025) introduced Bitswap 2.0 with much lower bandwidth overhead. Public Gateways: ipfs.io, dweb.link (Protocol Labs), cf-ipfs.com (Cloudflare 2022-2024 sunset, replaced by 4everland.io, w3s.link via Storacha). Gateways translate ipfs://CID → HTTP, enabling browser access. Lacks: Built-in incentive layer (data is opportunistically cached, garbage-collected after 7 days default). Persistence requires explicit pinning. Stats: ~250,000 public nodes (transient), ~1 EiB of data referenced (most under-replicated), Kubo Docker pulls 100M+, gateway traffic ~5-10 PB/month aggregated. Pinning services: Pinata (commercial, $20-$1000/mo tiers, 500K+ users), Storacha (Protocol Labs free 5GiB then paid, UCAN-based, formerly web3.storage), Fleek (Filecoin-backed pinning + edge CDN), Filebase (S3-compatible IPFS/Sia bridge), 4everland (Asia-focused).

2. Filecoin

Launched: October 2020 (Protocol Labs, Juan Benet). The IPFS incentive layer, designed in tandem since 2014. Architecture: Layer-1 blockchain (EC consensus — Expected Consensus, leader election via VRF + power-weighted, ~30s block time). Storage providers (formerly miners, rebranded SP 2024) commit physical disk via PoRep (initial sector seal) and ongoing PoSt (every ~24h prove sectors still hold data). Storage deals are signed off-chain between clients and SPs, then included on-chain for slashing accountability. FVM (Filecoin Virtual Machine): Activated March 2023 — EVM-compatible smart contract layer atop Filecoin’s actor model (FIPs 030/032). Unlocked DataDAOs (collective storage purchasing), perpetual storage contracts (auto-renewing deals), retrieval markets, FIL liquid staking (Glif, Stader), cross-chain bridges (Axelar, LayerZero). Filecoin Plus: DataCap notary program — verified clients (academic researchers, public archives, web3 protocols) receive 10x deal-weight multiplier, incentivizing SPs to store “useful” data over self-deals. Notaries (~40 globally) vet applications. Common Crawl, Internet Archive, Project Gutenberg are major DataCap recipients. Retrieval: Historically slow (must unseal sectors, 10-30min). Saturn (2023+) is Filecoin’s L1 retrieval CDN — incentivized cache nodes serve hot CIDs in <500ms. Project FWS (Filecoin Web Services) bundles S3-compatible APIs atop deals + Saturn. Stats: ~21 EiB committed capacity, ~2-3 EiB active client data, 3,800+ active storage providers globally, ~500-700M FIL circulating, market cap fluctuating $1.5-$3B 2025.

3. Arweave

Launched: November 2018 (Sam Williams, Sevki Hasirci). Different philosophical bet: permanence over churn. Architecture: Blockweave — each new block references the previous block plus a randomly selected historical “recall block.” Miners must serve random historical data to mine new blocks → economic incentive to retain complete history. SPoRA (Succinct Proofs of Random Access) replaced earlier consensus 2020-2021, requiring miners prove random access reads on historical chunks within deadline. Endowment Model: Single up-front payment in AR token (priced per byte at deal time) is split: small portion to immediate miners, majority deposited into a programmatic endowment that pays miners forever. Pricing model assumes ~30% annual storage cost decline (Kryder’s Law); even pessimistic 0.5% decline yields ~200 years coverage. Permaweb: Browser apps deployed to Arweave (HTML/JS/CSS bundles) become permanent and addressable via gateway domains (arweave.net/TXID, ar.io permanode network 2024+ for decentralised gateways). Major permaweb apps: Mirror.xyz (essay publishing, $WRITENFTs),Akord(encryptedvaults),ArDrive(Dropbox-like).\ast \ast Bundlers(Bundlr\to Irys)\ast \ast :Layer-2transactionaggregation.Irys(formerlyBundlrNetwork,rebranded2024)packsthousandsofusersubmissionsintosingleArweavetransactions,paidinanyof14+tokens(ETH,SOL,MATIC,ATOM,etc.),settledtoAR.ReducedfrictionenabledNFTmetadataatscale—Solan{a}^{\prime }sMetaplex,polygonNFTprojectsuseIrys.\ast \ast Stats\ast \ast :150PBstoredasof2025,1,500activeminingnodes,endowmentfundestimated$250-$350MdependingonARprice,ARmarketcap$400M-$1.5B range. Notable archives: Wikipedia mirror, Internet Archive partial replication, Twitter/X data scraping projects.

4. Storj (Decentralized Cloud Storage / DCS)

Launched: 2018 (initial), production network 2019 (“Tardigrade”), rebranded to “Storj DCS” 2021. Architecture: Three-tier system: Uplink (client encrypts + erasure-codes data into 80 pieces, any 29 reconstruct), Storage Nodes (24,000+ globally, run by hobbyists/datacenters, paid in STORJ ERC-20 token monthly via micropayments), Satellites (metadata coordinators — currently Storj Inc. runs the dominant satellite; community satellite federation in beta 2025). Encryption: All data client-side encrypted by default (Storj never sees plaintext). Uses path-encrypted hierarchies enabling sharing without server trust. S3-Compatibility: Drop-in replacement for AWS S3 SDK via Storj gateway — significant adoption among indie hackers, video creators, scientific data archivers. Pricing $4/TB/monthstorage+$7/TB egress (vs AWS S3 $23storage+$90 egress — 4-13x cheaper). Stats: ~30 PB stored, 24,000 node operators in 100+ countries, customer count 50,000+, notable: Atempo backup, Acronis, Akamai/Linode partnership (Linode object storage backed by Storj 2023+).

5. Sia / Skynet

Launched: 2015 (Nebulous Labs, David Vorick, Luke Champine). Among the earliest decentralised storage protocols, predates Filecoin by 5 years. Architecture: Renter-host file contracts on Sia’s blockchain. Files split, encrypted client-side, redundantly distributed to multiple hosts. Hosts post collateral; contracts include merkle-proof storage challenges. Native token Siacoin (SC). Renterd: Rewritten renter daemon (2023-2024) — modernised the user experience, S3-compatible API, used by Filebase as alternative to IPFS-backed storage. Skynet: Launched 2020 as Sia’s “hot retrieval” CDN for web apps, sunset May 2022 due to financial unsustainability (Skynet Labs shut down). The Sia core network continues, refocused on cold storage / archival. Stats: ~2 PB used capacity, ~600 active hosts, SC market cap $50-150M range.

6. Swarm

Launched: Mainnet June 2021 (Swarm Foundation, originally Ethereum Foundation incubation 2015+). Architecture: Ethereum companion network. Designed for the “world computer’s” storage layer alongside Ethereum compute and Whisper messaging (Whisper deprecated). Native token BZZ issued via bonding curve (price determined by reserve ratio). Bee client (Go) is the reference implementation. Postage Stamps: Prepaid storage receipts. Users buy postage stamps (BZZ) to “stamp” chunks; nodes serve only stamped content. Stamps have batch IDs and amounts — refilling extends storage duration. Single-Owner Chunks (SOC): Mutable references — content addressed by owner public key + index. Enables feeds, mutable state on top of immutable underlying chunks. Stats: ~30 PB capacity, ~10K Bee nodes, BZZ market cap $20-80M. Adoption lighter than Filecoin/Arweave but consistent Ethereum-native deployments (ENS profile data, some L2 rollup data availability experiments).

7. Walrus (emerged 2024-2025)

Launched: Devnet October 2024, mainnet March 2025 (Mysten Labs, Sui Foundation). Architecture: Sui-blockchain-coordinated decentralised storage. RedStuff encoding — novel 2D Reed-Solomon scheme using primary + secondary slivers achieving ~5x replication overhead (vs 25x naive replication) for similar Byzantine fault tolerance. Walrus storage nodes stake WAL tokens; data is referenced by blob ID with Sui smart contracts managing storage lifecycle, deletion (allowed unlike Arweave), expiry, and payments. Aggregators / Publishers: Like IPFS gateways but native — aggregators serve reads, publishers handle writes. Decentralised set of operators. Stats: As of mid-2025: ~150 storage nodes mainnet, several PB committed, WAL launched March 2025. Integrated with Sui ecosystem (NFTs, AI dataset distribution, DeepBook). Early but fast-growing.

8. Codex (Ethereum 2 Native, Testnet 2024-2025)

Background: IFT (Status / Logos / Codex) collective, addressing data availability for Ethereum L2 rollups + general-purpose decentralised storage. Architecture: Erasure-coded chunks, KZG polynomial commitments enabling efficient data availability sampling (DAS) — light clients can verify availability by sampling small random subsets. Aligned with Ethereum’s post-Cancun blob (EIP-4844) and post-Verkle roadmap. Status: Testnet through 2024-2025, mainnet expected 2026. Not yet production. Significant because of its Ethereum-native positioning and data-availability-sampling cryptography (relevant to rollup ecosystems).

9. Other Networks (Briefer Coverage)

• Crust Network (2020, Substrate chain): IPFS pinning with on-chain incentives via MPoW (Meaningful Proof of Work proving SGX-attested storage). Native CRU token.
• Shadow Drive (2022, Solana): Programmable user-managed buckets backed by Genesys Go infrastructure. SHDW SPL token. Tight Solana ecosystem integration.
• AIOZ Network: Decentralised CDN + storage on its own Cosmos-based chain. AIOZ token. Focus on streaming/AI inference delivery.
• 0Chain / Züs: Allocation-based storage with locked tokens. Niche adoption.
• BTFS (BitTorrent File System) by Tron/Justin Sun: IPFS fork with BTT token, smaller technical contribution, marketing-heavy.

CDN-Style Bridge Services (Web2 ↔ Web3)

Direct consumer/developer adoption of raw IPFS/Filecoin/Arweave is rare; most usage flows through bridge services that hide the cryptographic complexity behind familiar HTTP APIs:

• Storacha (formerly web3.storage, Protocol Labs, rebranded 2024): UCAN-based authorization (users hold did:key identities, delegate write capabilities cryptographically), free tier 5 GiB, paid scaling. Backs both IPFS hot serving + Filecoin cold archival. Successor to nft.storage which sunset June 2024.
• Pinata: Established 2018, IPFS pinning incumbent. ~500K users, sub-$20/mo entry pricing, “Dedicated Gateways” for predictable retrieval, integrated with major NFT marketplaces.
• Fleek: Static site / Next.js / Edge functions deployed atop IPFS + Filecoin + Internet Computer + Arweave. ~50K projects. Competes with Vercel for web3-native developers.
• Filebase: S3-compatible API translating to IPFS / Sia / Storj behind the scenes — “Web3 S3.” Drop-in for AWS SDK code.
• 4everland: Asia-focused (Hong Kong/Singapore) Web3 hosting with IPFS/Arweave/Internet Computer/Greenfield support. ~150K users.
• Irys (formerly Bundlr Network, rebranded 2024): Arweave’s L2 bundler — bundles thousands of user uploads into single Arweave transactions, accepting payment in 14+ tokens (ETH, SOL, MATIC, AVAX, BNB, ATOM, NEAR, …). Dominant pathway for NFT metadata on Solana/Polygon/Avalanche.

Standards & Interoperability

Five specifications enable interoperability across networks and clients:

• CID (Content Identifier): Self-describing identifier <multibase><cid-version><multicodec><multihash>. Most modern CIDv1 use base32 + dag-pb + sha2-256 → bafybeig.... Specified by IPFS team + Multiformats working group.
• IPLD (InterPlanetary Linked Data): Data model abstracting “linked data over content-addressed substrates.” Codecs include DAG-CBOR (CBOR with CID links), DAG-JSON, DAG-PB (the legacy IPFS format). IPLD Schemas define types/structures. Used by Filecoin, IPFS, Ceramic, Textile.
• CAR (Content Addressable aRchive): Single-file format packaging a DAG plus its root CIDs. CARv1 (2019) simple; CARv2 (2022) adds in-file indices for random access. Used for transport (Filecoin deal preparation, Storacha uploads, IPFS migrations) — analogous to .tar but with merkle integrity.
• UCAN (User-Controlled Authorization Networks): DID-based capability tokens. A user signs a UCAN token granting another DID (a service) limited rights (e.g., “store under prefix /uploads/alice”). Tokens can be re-delegated, time-bound, revoked. Adopted by Storacha, Fission, Bluesky/AT Protocol auth. W3C / IETF draft tracks 2024-2025.
• Multiformats: Family of self-describing serialisations (multihash, multiaddr, multicodec, multibase) underlying CIDs and libp2p. Maintained by multiformats.io community.

Use Cases / Major Families

Decentralised file storage’s economic justification flows from use cases that intolerate cloud-storage trust assumptions:

NFT Metadata Persistence

The single largest by-volume use case. Centralized NFT metadata (https://example.com/token/123.json) is vulnerable to domain expiry, server shutdown, malicious mutation. Standards (ERC-721 metadata JSON, ERC-1155) increasingly require ipfs://CID or ar://TXID URIs. OpenSea, Magic Eden, Blur, Rarible, all major marketplaces resolve both schemes. Adobe Behance integration (2022) auto-pins to IPFS. Damage from centralized failures (Mt. Gox metadata loss, NFT-domain rugpulls 2021-2022) created industry-wide migration pressure. ~10M+ NFT collections rely on decentralised storage as of 2025.

Blockchain Data Archival

Ethereum full archive nodes consume 18+ TB; Bitcoin full archive ~600 GB; Solana ledger ~400 TB historical. Decentralised storage hosts:

• ar.io permanodes: Distributed Arweave gateway operators serving Ethereum state snapshots, Bitcoin block data.

• Filecoin DataCap: Internet Archive 500+ PiB across multiple SPs, Common Crawl 320 TB crawl data, Project Gutenberg, OpenStreetMap diffs.

• The Graph + IPFS: Subgraph manifests + schemas live on IPFS, retrieved by indexers building queryable APIs over chain data.

  Censorship-Resistant Publishing

• Mirror.xyz: Essays + posts written by ~500K authors, content stored on Arweave (immutable, signed by Ethereum keys), monetisable via $WRITE NFTs. Wikipedia mirrored to IPFS during 2017 Turkey block.

• DistributedPress / Hypha: WordPress-alternative federated publishing toolchain pushing to IPFS + ActivityPub.

• CryptpAd: Encrypted collaborative documents (CRDT-based), optionally pinned to IPFS for resilience.

  Video Streaming

• Livepeer + IPFS: Decentralised transcoding network outputs HLS chunks pinned to IPFS, retrieved via gateways/CDN.

• Theta Network + EdgeCacher: Decentralised video CDN with token-incentivised edge caches. Partnerships with Samsung, Sony, Lionsgate (limited content).

• DTube / Odysee: BitTorrent + IPFS-backed video hosting, alternative to YouTube. ~10M active users Odysee.

  AI Dataset Distribution

• Hugging Face: Partial IPFS mirroring of popular datasets (LAION-5B, RedPajama) reducing single-CDN bandwidth costs.

• Filecoin Foundation Common Crawl: 320 TB monthly crawl pinned by FIL+ verified clients.

• Bacalhau (Protocol Labs): “Compute over Data” — run jobs co-located with IPFS/Filecoin data avoiding egress costs. Enables verifiable inference on decentralised datasets.

• The Stack (BigCode): 6 TB code corpus partially on IPFS/Filecoin for resilient research access.

• LAION-5B archival: 5.85B image-text pairs (~240 TB) mirrored across Filecoin SPs after the original CDN faced bandwidth pressure 2023, demonstrating decentralised storage’s strategic role in safeguarding open AI training datasets.

• Web2-to-Web3 dataset bridges: Ocean Protocol’s compute-to-data marketplace, Numerai’s encrypted financial data tournaments, Genomes.io’s privacy-preserving genomic datasets — all leverage decentralised storage for tamper-evident dataset provenance critical for AI reproducibility.

  Scientific Data and Open-Access Research

• Internet Archive partnership with Filecoin: 500+ PiB of historical web crawls, books, audio, video archived via FIL+ DataCap deals 2021-onwards. Hedge against single-operator Internet Archive failure.

• NASA / Earth observation: Pilots placing Landsat / Sentinel imagery on Filecoin for scientific community redundancy beyond AWS Open Data sponsorship.

• OpenStreetMap: Periodic full-planet dumps (~70 GB compressed) replicated via IPFS/Filecoin; community-controlled redundancy.

• Pre-print archives: Some bioRxiv / arXiv mirroring experiments on Arweave for permanent citation-stable references.

  Software Distribution and Reproducible Builds

• NPM / PyPI mirrors: Experimental IPFS mirrors for resilience against single-registry compromise (left-pad incident class).

• Docker / OCI image distribution: nix-store IPFS integration, IPDR (IPFS-backed Docker Registry) for content-addressed container distribution avoiding registry centralisation.

• Operating system images: Arch Linux, NixOS occasional IPFS distribution. Tor Browser shadow distribution via IPFS.

• Reproducible builds: Content addressing aligns with reproducible build hashes (Nix, Bazel, Guix) creating natural integration paths.

  Decentralised Identity and Personal Data

• Ceramic / ComposeDB: User-controlled mutable streams atop IPFS. Identity profiles, social graphs, application state.

• Bluesky / AT Protocol: PDS (Personal Data Server) architecture with content-addressed blobs and CAR-file repository backups. 30M+ users by mid-2026 validating consumer-scale content-addressing.

• Solid pods + IPFS bridges: Tim Berners-Lee’s Solid project integrates with decentralised storage backends for personal data vaults.

Failure Modes and Operational Realities

Critical to honest evaluation — decentralised storage has distinctive failure modes:

• Pinning Churn: Single-pinned IPFS data vanishes when the pinning operator stops paying. Pinata’s free-tier deprecation 2024, web3.storage migration to Storacha with grandfathered limits, occasional small-pinner shutdowns have repeatedly orphaned NFT metadata. Industry survey (Ribbit Capital 2023) estimated 25% of NFT collections from 2021 had at least partial metadata loss within 18 months.
• Retrieval Latency: Cold Filecoin sectors require unsealing (10-30 minutes). DHT walks for unindexed CIDs can take 30 seconds. IPFS public gateway tail latency p99 1-5 seconds vs CloudFront p99 <100ms. Saturn cache layer (2023+) addresses hot retrieval but cold-tail data remains slow.
• Data Availability vs Persistence Confusion: IPFS is not storage. CIDs without pinning are forgettable. The mental model “I put it on IPFS” is misleading — must pin or deal. Education gap among developers leads to systematic data loss.
• Cryptoeconomic Volatility: Filecoin storage prices fluctuate with FIL token price (sometimes 5x within months). Arweave’s endowment model depends on AR maintaining real-terms value over decades. Bear markets (2022, 2024 mini-cycles) compressed storage provider economics, with marginal SPs exiting; impact on long-tail data unclear.
• Gateway Centralization: Most usage flows through ~5 gateways (ipfs.io, dweb.link, w3s.link, Pinata, Cloudflare-historical). Operationally similar to cloud dependency despite cryptographic decentralisation. ar.io’s permanode network (2024+) directly addresses this for Arweave; IPFS gateway federation (Service Worker IPFS, Helia in-browser) progressing slowly.
• Regulatory Uncertainty: GDPR right-to-erasure conflicts with content-addressed immutability. UK/EU regulators have flagged this; Walrus and others explicitly support deletion to mitigate. UK Online Safety Act 2023 imposes content moderation obligations on user-to-user services — implications for IPFS gateway operators uncertain. EU Data Act and DMA acknowledge but do not yet specifically regulate decentralised storage.
• Content Moderation: Permaweb models (Arweave) cannot remove illegal/abusive content from network history. Practical mitigations: gateway-level filtering (cf-ipfs.com bad-bits list, Cloudflare gateway blocklists), publisher-level encryption (content cannot be served without key destruction = de facto erasure), but tension remains between censorship-resistance ethos and child-protection / copyright enforcement.
• Geographic Concentration Despite Decentralisation: Filecoin storage provider concentration in China (historical pre-2024) and US/EU created jurisdictional risk. Storj’s 100+ country distribution is more even. Network-level decentralisation does not automatically yield jurisdictional diversity.
• Software Diversity / Client Risk: Kubo dominance (95%+ of IPFS public nodes) is a monoculture risk. Helia (JS), iroh (Rust), rust-ipfs improve diversity but still nascent. Filecoin Lotus reference client dominance similar; Forest (Rust), Venus (Go alt) exist but minor.
• Quantum Threat Horizon: SHA-256 (CIDs) and current Ed25519/RSA (libp2p, FIL signatures, Arweave wallets) face long-term quantum risk. Migration to post-quantum primitives (Dilithium/Falcon, SHA-3) required by 2030-2035 estimates. Permanent-storage networks particularly exposed given century-scale guarantees.

Academic Context: Foundational Research

Decentralised file storage rests on a multi-decade research lineage spanning theoretical computer science, cryptography, and distributed systems engineering. Below we summarise the contributions that directly underpin contemporary networks.

Distributed Hash Tables (1997-2004)

• Plaxton, Rajaraman & Richa (SPAA 1997) introduced prefix-routing trees foundational to overlay routing.

• Stoica et al. (SIGCOMM 2001) Chord: O(log n) consistent-hashing-based routing. Established baseline complexity bounds.

• Maymounkov & Mazières (IPTPS 2002) Kademlia: XOR-metric DHT with parallel queries and learning behaviour. Powers libp2p DHT in IPFS/Filecoin/Polkadot/Ethereum 2.

• Rowstron & Druschel (Middleware 2001) Pastry: Prefix-routing variant; influenced past DHT-based storage (PAST). Sheffield’s heritage links to this lineage.

• Ratnasamy et al. (SIGCOMM 2001) CAN (Content-Addressable Network): d-dimensional torus routing; less influential commercially but academically foundational.

  Content-Addressed Storage (1999-2010)

• Quinlan & Dorward (USENIX 2002) Venti: Plan 9 archival storage system; first widely-known content-addressed block store.

• Mazières et al. (SOSP 1999) SFS (Self-Certifying File System): Self-verifying file system names — direct precursor to CID concept.

• Bonomi et al. (1986+) Hash-based filesystems theoretical groundwork.

• Hewlett-Packard’s “Centera” (commercial 2003) introduced content-addressed enterprise storage but proprietary, not P2P.

  Peer-to-Peer Systems (2001-2010)

• Cohen (2001) BitTorrent: Tit-for-tat block exchange. Direct conceptual ancestor of Bitswap.

• Napster (1999), Gnutella (2000), eDonkey/Overnet (2002), Freenet (2000) — earlier P2P precursors; Freenet’s content-addressing and routing-without-central-server design particularly influential.

• Wilcox-O’Hearn & Warner (StorageSS 2008) Tahoe-LAFS: First production-grade erasure-coded distributed storage with cryptographic capability-based access. Influenced Storj architecture directly.

  Cryptographic Storage Proofs (2007-2019)

• Juels & Kaliski (CCS 2007) PORs (Proofs of Retrievability): Sentinel-based proofs that a server retains data.

• Ateniese et al. (CCS 2007) PDP (Provable Data Possession): Sampling-based efficient verification; foundation of Filecoin’s PoRep formalisation.

• Dziembowski et al. (CRYPTO 2015) Proofs of Space: Time-memory trade-offs for proving disk allocation, foundational for Chia / Spacemint.

• Fisch et al. (IACR 2018) Scaling Proof-of-Replication for Filecoin Mining: Stacked Depth Robust (SDR) hashing enabling sector-scale PoRep.

• Fisch (EUROCRYPT 2019) Tight Proofs of Space and Replication: Compact storage proofs.

• Ben-Sasson, Chiesa, Goldberg, Tromer (2013+) groundwork on succinct non-interactive arguments (zk-STARKs/SNARKs) enabling on-chain verification of storage proofs at low gas cost.

  Erasure Coding for Distributed Storage

• Reed & Solomon (1960) original Reed-Solomon codes.

• Plank (1997) tutorial making RS implementation accessible to systems community.

• Rashmi et al. (SIGCOMM 2014) “Hitchhiker’s guide” optimising reconstruction bandwidth; Facebook-led work directly relevant to modern distributed storage erasure schemes.

• Sathiamoorthy et al. (VLDB 2013) Xorbas / locally-repairable codes — foundation for Storj’s reconstruction.

• Pittarello et al. (Mysten Labs 2023) RedStuff 2D Reed-Solomon for Walrus achieving ~5x overhead.

  Protocol Whitepapers

• Benet (2014) IPFS arXiv:1407.3561 — the founding paper.

• Vorick & Champine (2014) Sia — earliest crypto-economic decentralised storage.

• Protocol Labs (2017) Filecoin — formalised PoRep + PoSt.

• Williams & Diao (2018) Arweave Yellow Paper — endowment model + blockweave.

• Trón (2019) Book of Swarm — Ethereum-companion storage.

• Miller et al. (IEEE S&P 2014) Permacoin: academic precursor demonstrating Bitcoin work could fund data preservation, foreshadowing Arweave’s design.

• Mysten Labs (2024) Walrus paper — Sui-native storage.

  Emerging Research Directions

• Data availability sampling (DAS) for L2 rollups (Al-Bassam, Sonnino, Buterin 2021) — Codex’s research base.

• Verifiable computation over content-addressed data (Bacalhau, Lilypad Network) — emerging “compute-over-data” thesis.

• Privacy-preserving decentralised storage (Nucypher proxy re-encryption, Lit Protocol decentralised access control, Fhenix FHE-based encrypted compute over storage).

• Post-quantum migration paths for permanent storage networks — active 2025-2026 research.

Current Landscape (2026)

As of Q1-Q2 2026 the field has stabilised into a clear hierarchy with several recent developments:

• Filecoin (~21 EiB committed, ~3 EiB active): FVM ecosystem matured — Glif liquid staking ~50M FIL, retrieval markets via Saturn delivering p50 <500ms, FilOz organisational restructure 2025. Filecoin Foundation in London (Filecoin Foundation for the Decentralized Web) coordinating standards work.
• Arweave (~150 PB): Ao computation layer (2024 launch by Sam Williams) extending Arweave from storage to “permanent computation.” ar.io permanode network for decentralised gateway resilience (replacing arweave.net single-operator risk). Endowment health audits 2024-2025 positive.
• Storj DCS: Akamai/Linode partnership matured (Linode Object Storage backed by Storj for select tiers). 30 PB+ used, 24K nodes. Atempo, Acronis, MASV (large-file delivery) production customers.
• Walrus (mainnet March 2025): Mysten Labs flagship. WAL token TGE March 2025. RedStuff encoding proved out at production load. AI dataset commons partnerships (Sui Foundation funding).
• Codex: Testnet matured through 2025. Mainnet expected late 2026. Strategic positioning for Ethereum data availability post-Verkle.
• Saturn (Filecoin retrieval CDN): 4000+ cache nodes globally as of 2026, serving 1 PB+ monthly with sub-second p50 retrieval. Bridges decentralised storage with web-grade UX.
• AT Protocol / Bluesky: While not file storage per se, Bluesky’s PDS (Personal Data Server) architecture with content-addressed blobs and CAR-file backups validates content-addressing patterns at consumer scale (30M+ users by mid-2026).
• Storacha (rebranded web3.storage, 2024 onwards): UCAN-native authorization, mature SDK ecosystem, ~100K developers.
• EU Regulatory Posture: EU Data Act (in force 2025) explicitly recognised decentralised storage as a legitimate compliance pathway for some interoperability requirements; GDPR-erasure debates ongoing.
• Token Markets 2026: FIL $3-5Bmcap,AR$300M-$1B,FIL+AR+STORJ+BZZ+SC+SHDW+WALaggregate$5-8B.

UK Context: Academic and Industrial Centres

The UK plays an outsized role in decentralised storage standards, research, and industry coordination relative to its size.

Academic Institutions

Imperial College London (Department of Computing):

• Distributed Systems Research Group: Peter Pietzuch’s group has published on decentralised storage performance, byzantine fault tolerance under workload. Imperial hosts the UK end of various Protocol Labs research grants on libp2p performance.

• Centre for Cryptocurrency Research and Engineering (IC3 affiliate): Cross-disciplinary work on storage proofs, zk-SNARK efficiency relevant to Filecoin’s PoRep.

• Industry collaboration: Filecoin Foundation London (UK charity arm), regular workshops with Protocol Labs.

  University of Edinburgh (School of Informatics, Blockchain Technology Lab):

• Aggelos Kiayias’s Blockchain Technology Lab (IOG/Cardano-affiliated) publishes on storage proofs, proof systems applicable to decentralised storage. Edinburgh hosts the Programmable Smart Money community for FVM developers.

• PhD research on retrievability proofs (proof-of-data-possession variants) directly relevant to Filecoin/Storj.

  University of Cambridge (Department of Computer Science and Technology):

• Centre for Alternative Finance maintains the Cambridge Bitcoin Electricity Consumption Index and has published on Filecoin energy/storage economics.

• Cryptocurrency research group analyses storage-network token economics, incentive design.

  UCL Centre for Blockchain Technologies (CBT):

• One of Europe’s largest blockchain academic programmes. Publishes on distributed storage governance, regulatory implications of decentralised storage. PhD theses on Filecoin economics, Arweave permanence guarantees.

• Industrial advisory board includes Filecoin Foundation, Ethereum Foundation contributors.

  University of Oxford (Department of Computer Science):

• Formal verification work on smart contracts (relevant to FVM auditing). Quantum computing research with implications for long-term cryptographic security of permanent-storage schemes (Arweave 200-year horizon).

  Industry and Standards Bodies

  Filecoin Foundation (London): Headquartered in London, registered UK charity. Coordinates research grants, standards work (IPLD, IPFS specifications), DataCap notary programme governance. Marta Belcher (CEO) coordinates internationally; London office handles European policy engagement.

  Protocol Labs London: Engineering presence in London (libp2p, IPFS Kubo development). Hosts annual IPFS Camp / IPFS Thing events. Recruits substantially from Imperial/UCL/Cambridge.

  Bluesky / AT Protocol (UK contributors): Significant UK engineering contributors to AT Protocol, which uses content-addressed blobs and CAR files — adjacent to decentralised file storage standards.

  Fluence Labs (Edinburgh office): Decentralised serverless computation with IPFS/libp2p integration. UK engineering hub.

  Northern English Industrial Hubs

  Manchester:

• Manchester Blockchain Hub (University of Manchester + Innovation Greater Manchester): Several SMEs experimenting with Arweave/IPFS for NHS data archival pilots (encrypted patient records pinned to IPFS, keys on Ethereum DIDs). N8 partnership research grants.

• MediaCityUK (Salford): BBC R&D has explored IPFS for archive distribution (BBC Archive PoC 2023-2024, content-addressed BBC Sounds backups). Limited production deployment but ongoing.

  Leeds:

• University of Leeds (School of Computing): Research on decentralised data archival applied to scientific datasets (LIDA — Leeds Institute for Data Analytics). Filecoin DataCap allocations for academic datasets.

• Leeds Beckett: Industry workshops with local fintech on decentralised storage for KYC document retention (long-term immutable audit trails).

  Sheffield:

• University of Sheffield (Department of Computer Science): P2P systems heritage (predecessor research on Pastry/PAST DHTs informed modern Kademlia implementations). Current research on incentive-compatible storage proofs.

• Sheffield AMRC (Advanced Manufacturing Research Centre): Pilots on tamper-evident manufacturing data archival via Arweave for Rolls-Royce, BAE Systems supply chain projects.

  Newcastle:

• Newcastle University (School of Computing, Open Lab): Digital civics research includes decentralised data sovereignty work — community-controlled storage as alternative to corporate cloud for civic data.

• Digital Catapult North East: SME acceleration including web3 startups exploring Filecoin/IPFS integration; modest cohorts.

  UK Government / Public Sector Engagement

• UK AI Safety Institute / DSIT: Decentralised storage considered for AI dataset provenance / model card persistence (early-stage policy discussions 2025).

• TNAS (The National Archives): Pilot research on decentralised storage as archival redundancy mechanism (Filecoin DataCap allocation under evaluation).

• Ofcom: Watching Bluesky / AT Protocol developments for Online Safety Act implications — by extension, decentralised storage’s content moderation challenges.

• NHS Digital / NHSX: Limited experimentation with patient-data anchoring on Ethereum + IPFS for tamper-evident audit trails. No production deployments as of 2026; data residency concerns favour UK-domiciled storage operators.

• British Library: Long-term digital preservation working group has assessed Arweave and Filecoin as supplementary archival mechanisms; institutional adoption pending governance frameworks.

• JISC (HE/FE network): Research data archival pilots exploring Filecoin DataCap for university dataset replication, particularly for funder mandates requiring long-term access.

  UK Startup / SME Ecosystem

• Fission (originally Vancouver, now distributed with UK contributors): UCAN-based personal data infrastructure atop IPFS. Influential in UCAN specification work.

• Estuary (acquired by Protocol Labs): UK contributors to Filecoin deal-making automation.

• Spheron / Akash Network UK partners: Decentralised compute partnering with decentralised storage for hybrid web3 deployments.

• Outlier Ventures (London): Web3 accelerator with several decentralised-storage portfolio companies including data DAO infrastructure.

• DFINITY / Internet Computer UK team: Adjacent to decentralised storage with on-chain blob storage primitives; ecosystem-adjacent rather than direct competitor.

Future Directions (2026-2030)

Five trajectories will shape the next 5 years:

1. Hot/Cold Convergence

Filecoin’s retrieval markets (Saturn), Walrus’s aggregator/publisher pattern, Storj’s continued S3-compatibility, Fleek’s edge deployment all push toward sub-second retrieval indistinguishable from cloud CDNs. By 2028 expect <200ms p50 retrieval as table-stakes; competitive parity with Cloudflare R2/AWS S3 for read-heavy workloads.

2. AI Dataset Commons

AI training corpus economics (10-100PB per major model, $50-500M cloud egress costs) create strong incentives for decentralised hosting. Filecoin Foundation’s data commons (Common Crawl, LAION-5B mirrors, Hugging Face partial replicas), Arweave’s permanent AI history (model weights, training logs), and Walrus’s Sui-native AI integrations will mature. Expect dedicated FIL+/DataCap programmes for AI safety datasets, model cards, RLHF preference datasets by 2027.

3. Data Availability for Rollups

Ethereum L2 rollups (Optimism, Arbitrum, Base, zkSync, Starknet) generate massive transaction data. Current EIP-4844 blobs (~32 MB/block, 18-day pruning) are insufficient long-term. Codex (Ethereum-native), Celestia (existing data-availability chain), and EigenDA (restaking-secured) compete. Decentralised file storage networks (Filecoin, Walrus) may capture some long-term archival of rollup history.

4. Cryptographic Upgrades

Post-quantum signatures (Dilithium/Falcon NIST standardisation 2024) will be required for permanent-storage networks (Arweave’s 200-year horizon, Filecoin’s century-deal contemplations). Expect migration plans 2027-2029. zk-SNARK proof systems for storage (PoRep) will continue shrinking: Filecoin’s PoRep evolution from Groth16 to Halo2 to STIR/Plonky3 cuts seal compute time.

5. Regulatory Maturation

GDPR right-to-erasure vs content-immutability tensions will be resolved through architecture (encryption + key destruction = de facto erasure, adopted by Storj, Walrus’s deletable blobs), not by exempting protocols. UK Online Safety Act / EU Digital Services Act compliance will require gateway operators (Cloudflare-style) to implement content takedown — putting pressure on permaweb/Arweave’s permanence in particular. Standards bodies (IETF UCAN working group, W3C DID/VC, NIST DLT standards) will formalise interoperability by 2028.

6. Hybrid Architectures (Cloud + Decentralised)

Pure decentralised deployments remain rare in enterprise; hybrid patterns dominate practical adoption. By 2028-2030 expect standardised architectures placing hot tier on cloud (S3/R2/GCS, <50ms p99), warm tier on decentralised CDN (Saturn, AIOZ, Fleek, 100-500ms), cold tier on Filecoin/Arweave (minutes-hours retrieval acceptable). Such architectures deliver censorship-resistance and provenance for the cold tier while preserving familiar latency for active reads — Coinbase, Kraken, OpenSea already operate hybrid metadata pipelines.

7. AI-Native Storage Primitives

Storage networks are evolving primitives specifically targeting AI workloads: model card permanence, training-data provenance with cryptographic dataset hashes, RLHF preference dataset archives, “AI bill of materials” attestations. Bacalhau (compute-over-data), Lilypad Network (decentralised inference), and Walrus’s Sui-native AI integrations will mature. Expect 2027-2028 standardisation of dataset CID conventions for AI safety auditing.

Market Projections 2026-2030

• 2026 baseline: Filecoin 21 EiB committed / 3 EiB active, Arweave 150 PB, Storj 30 PB, IPFS public ~1 EiB referenced (mostly under-replicated), Walrus 5-10 PB, Swarm 10 PB. Aggregate token mcap $5-8B.
• 2028 projection: Filecoin 50-80 EiB / 10-15 EiB active, Arweave 500 PB-1 EB, Storj 100 PB+, Walrus 100 PB+, Codex 10-50 PB (post-mainnet). Aggregate mcap $20-40B contingent on crypto cycle.
• 2030 projection: Decentralised storage capturing 3-5% of global object storage market ($50-100B TAM), driven primarily by AI data commons, NFT/metadata permanence, archival workloads. Cloud-equivalent retrieval performance achieved. Major AI labs run hybrid (cloud hot + decentralised cold) storage strategies.
• Adjacent market drivers: Sovereign data residency regulations (EU Data Act, India DPDPA, China data localisation) create demand for non-US-cloud storage with cryptographic provenance — favouring decentralised solutions. Edge computing convergence (5G/6G, latency-sensitive AR/VR) drives decentralised CDN adoption.

Comparison with Centralised Alternatives

Direct comparison clarifies trade-off space relative to incumbent cloud and CDN services:

Property	Cloud (S3/R2/GCS)	CDN (Cloudflare/Akamai)	Decentralised (Filecoin/IPFS/Arweave/Storj)
Durability	11 nines (operator-claimed)	N/A (cache)	11+ nines (cryptographic)
Read latency p50	50-200ms	10-50ms (edge)	100ms-30s (varies)
Egress cost	$0.08-$0.12/GB (AWS), $0 (R2)	Variable	$0.007-$0.025/GB (Storj), free (IPFS)
Censorship resistance	Operator-dependent	Operator-dependent	Strong (Arweave/Filecoin), Medium (Storj)
Permanence guarantees	Contractual	None	Cryptographic (Arweave) / Deal-bound (FIL)
GDPR erasure	Native	Native	Limited (Walrus/Storj yes, Arweave no)
Operational complexity	Low	Low	Medium-High
Tooling maturity	High (decades)	High	Medium (5-10 years)
Vendor lock-in	High	Medium	Low
Cost predictability	High	High	Medium (token volatility)

When decentralised storage wins: Archive workloads where permanence + censorship-resistance + cost matter more than tail latency (academic datasets, legal/regulatory records, NFT metadata, scientific provenance, censorship-vulnerable journalism). When data sovereignty and provider-agnosticism are strategic requirements.

When centralised wins: Latency-sensitive transactional workloads (database backups, real-time logs, video streaming primary tier), where SLA contracts with single counterparty satisfy compliance, where small data volumes (<TB) make cost differential immaterial.

Research and Literature

Foundational Whitepapers:

1. Benet, J. (2014). IPFS - Content Addressed, Versioned, P2P File System. arXiv:1407.3561. [Foundational protocol description]
2. Protocol Labs (2017). Filecoin: A Decentralized Storage Network. https://filecoin.io/filecoin.pdf. [PoRep + PoSt formalisation]
3. Williams, S., & Diao, V. (2018). Arweave: A Protocol for Economically Sustainable Information Permanence. https://www.arweave.org/yellow-paper.pdf. [Blockweave + SPoRA + endowment model]
4. Vorick, D., & Champine, L. (2014). Sia: Simple Decentralized Storage. https://sia.tech/sia.pdf. [Renter-host contract model]
5. Storj Labs (2018, revised 2022). Storj V3 Whitepaper: A Decentralized Cloud Storage Network. https://storj.io/storj.pdf. [Erasure-coded distributed cloud]
6. Trón, V. (2019). The Book of Swarm. https://www.ethswarm.org/the-book-of-swarm.pdf. [Swarm protocol comprehensive description]
7. Mysten Labs (2024). Walrus: An Efficient Decentralized Storage Network. https://docs.walrus.site/walrus.pdf. [RedStuff encoding, Sui integration]

Theoretical Foundations: 8. Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., & Balakrishnan, H. (2001). Chord: A scalable peer-to-peer lookup service for internet applications. SIGCOMM 2001, 149-160. DOI: 10.1145/383059.383071. [DHT foundations] 9. Maymounkov, P., & Mazières, D. (2002). Kademlia: A peer-to-peer information system based on the XOR metric. IPTPS 2002, 53-65. DOI: 10.1007/3-540-45748-8_5. [Kademlia DHT, used by libp2p] 10. Miller, A., Juels, A., Shi, E., Parno, B., & Katz, J. (2014). Permacoin: Repurposing Bitcoin work for data preservation. IEEE S&P 2014, 475-490. DOI: 10.1109/SP.2014.37. [Academic precursor to Arweave] 11. Wilcox-O’Hearn, Z., & Warner, B. (2008). Tahoe: The least-authority filesystem. StorageSS 2008, 21-26. DOI: 10.1145/1456469.1456474. [Erasure-coded distributed storage with capabilities] 12. Juels, A., & Kaliski, B.S. (2007). PORs: Proofs of retrievability for large files. CCS 2007, 584-597. DOI: 10.1145/1315245.1315317. [Theoretical foundation for storage proofs] 13. Ateniese, G., et al. (2007). Provable data possession at untrusted stores. CCS 2007, 598-609. DOI: 10.1145/1315245.1315318. [PDP framework underlying Filecoin PoRep]

Algorithmic / Implementation Papers: 14. Fisch, B., Bonneau, J., Greco, N., & Benet, J. (2018). Scaling proof-of-replication for Filecoin mining. IACR ePrint 2018/678. [PoRep optimisation] 15. Fisch, B. (2019). Tight proofs of space and replication. EUROCRYPT 2019, 324-348. DOI: 10.1007/978-3-030-17656-3_12. [Compact storage proofs] 16. Park, S., Pietrzak, K., Alwen, J., Fuchsbauer, G., & Gaži, P. (2018). SpaceMint: A cryptocurrency based on proofs of space. Financial Crypto 2018, 480-499. [PoS cryptography] 17. Rashmi, K.V., et al. (2014). A “hitchhiker’s” guide to fast and efficient data reconstruction in erasure-coded data centers. SIGCOMM 2014, 331-342. DOI: 10.1145/2619239.2626325. [Modern erasure coding] 18. Pittarello, F., et al. (2023). RedStuff: A 2D Reed-Solomon erasure code for decentralized storage. Mysten Labs technical report. [Walrus encoding]

Standards & Specifications: 19. IPFS Specifications (2024). https://github.com/ipfs/specs. [Comprehensive protocol specs] 20. IPLD Specifications (2024). https://ipld.io/docs/. [Data model + codecs] 21. CAR File Specifications (2022). https://ipld.io/specs/transport/car/. [CARv1/CARv2 transport format] 22. UCAN Specification v0.10 (2024). https://github.com/ucan-wg/spec. [DID-based capability tokens] 23. Multiformats (2024). https://multiformats.io. [Self-describing formats] 24. Filecoin Specification (2024). https://spec.filecoin.io. [Full protocol spec] 25. EIP-4844 (Buterin et al. 2024). Shard Blob Transactions. https://eips.ethereum.org/EIPS/eip-4844. [Ethereum DA blob spec, contextual to Codex]

Industry / State-of-the-Art Reports: 26. Filecoin Foundation (2024). Filecoin Network Statistics Dashboard. https://filfox.info / https://spacegap.github.io. [Live network metrics] 27. Messari (2025). Decentralized Storage State of the Art Q1 2025. https://messari.io. [Industry analysis] 28. Electric Capital Developer Report (2024). [Storage protocol developer activity rankings]

Synthesis and Critical Assessment

After a decade of production deployment the field has reached qualified maturity. Filecoin and Arweave have demonstrated exabyte-scale and century-horizon durability respectively; Storj has proven price-competitive S3-alternative status; Walrus’s 2025 mainnet validates Sui’s integrated approach to storage-as-primitive. Yet the category remains structurally smaller than cloud object storage (low single-digit percentage of global object storage TAM) and faces persistent tail-latency, regulatory, and developer-experience gaps. The decisive next phase (2026-2030) will be defined less by new networks than by maturation of three flywheels: (1) hot-retrieval performance reaching cloud parity (Saturn, Walrus aggregators, ar.io permanodes), (2) standardised hybrid architectures becoming default practice in regulated industries (finance archival, healthcare research data, government open data), and (3) AI-native storage primitives capturing the structural opportunity of model/dataset provenance. Whether decentralised storage becomes a 5% or 20% slice of the global storage market depends substantially on whether sovereignty regulation continues to drive cryptographic-provenance demand (current trajectory positive: EU Data Act, India DPDPA, China data localisation, US executive orders on AI provenance) and whether token volatility risk to storage-provider economics can be mitigated through stable-value derivative deals (FIL+ DataCap-style mechanisms generalised).

Metadata

• Last Updated: 2026-05-16
• Review Status: Comprehensive editorial review (Phase 6 enrichment)
• Verification: Network statistics cross-referenced against Filecoin Filfox / Arweave ar.io / Storj public stats / Walrus mainnet launch announcement
• Regional Context: UK academic institutions (Imperial, Edinburgh, Cambridge, UCL, Oxford) and industry coordination (Filecoin Foundation London, Protocol Labs London, BBC R&D MediaCityUK), Northern English innovation hubs (Manchester, Leeds, Sheffield, Newcastle) detailed
• Production-Ready: Complete OWL formal semantics, comprehensive network coverage (IPFS, Filecoin, Arweave, Storj, Sia, Swarm, Walrus, Codex, Crust, Shadow Drive, AIOZ), bridge services, standards (IPLD/CAR/UCAN), use cases, failure modes, future projections
• Domain: infrastructure (verified correct, not corrected)
• Authority Score: 0.87 (mature production networks with exabyte-scale deployments, academic foundations in DHTs/PoR/erasure coding, active standards work IPLD/UCAN, evidence-based industry projections)

Provenance

• domain-validation: verified-correct (infrastructure domain matches concept ontology, no correction required)