Swarm, under the hood: links for the Resilient Networking Hub

How does a request find its data?

Bee, the Swarm client, runs on libp2p. Nodes and chunks (4 KB pieces of a file) share one address space, and each node stores the chunks closest to its own address. Requests hop toward that address, a scheme called forwarding Kademlia.

Your nodeasks for a chunk

A peercloser to the chunk’s address

A closer peercloser still

The storerholds the chunk

Request: each hop goes to a peer closer to the chunk’s address. Chunk: comes back along the same path.

BeeThe Swarm client, in Go: networking on libp2p, routing, storage and retrieval.github.com/ethersphere/bee libp2pThe modular peer-to-peer networking stack Bee runs on.libp2p.io DISCHow Swarm stores and retrieves chunks across the network.docs.ethswarm.org/docs/concepts/DISC Kademlia in SwarmHow nodes find peers and route requests by address.docs.ethswarm.org/docs/concepts/DISC/kademlia Swarm Protocol SpecificationThe formal spec, for anyone writing a client or an integration.papers.ethswarm.org/p/swarm-protocol-spec

What happens when the publisher goes offline?

Once an upload has synced, it no longer depends on the publisher’s machine. It stays available as long as its postage stamp, the storage paid for with each upload, is valid.

Publisheruploads, then goes offline

Neighborhoodthe nodes closest to the chunk each keep a copy

Readerstill gets the file

Request from the reader. Chunk. The dashed arrow is the upload, made before the publisher went offline. Offline node. The rest of the neighborhood still has the chunk. If too many copies go missing, erasure coding’s spare chunks let the file be rebuilt from what’s left.

Postage stampsWhy content stays: every upload carries prepaid storage that runs down over time, like rent.docs.ethswarm.org/docs/concepts/incentives/postage-stamps The erasure coding paperThe research behind erasure coding in Swarm, including dispersed replicas for single chunks and the maths of each redundancy level.papers.ethswarm.org/erasure-coding.pdf Erasure coding in practiceHow to set a redundancy level on an upload, and how downloads rebuild missing chunks.docs.ethswarm.org/docs/develop/tools-and-features/erasure-coding

What if your browser is the node?

Freedom is a browser with a Swarm node built in, so it opens Swarm content straight from the network, with no gateway in between. It runs on desktop and mobile, and the team is now bringing streaming to Android.

Freedom BrowserA browser for the decentralized web, with Swarm, IPFS and ENS built in.freedom.baby Freedom source codeWhere the browser’s code lives today.github.com/solardev-xyz/freedom-browser Freedom HQThe project’s new GitHub home, where its repos are moving.github.com/freedom-hq Antstream for AndroidFreedom’s mobile streaming work: early Android builds.github.com/freedom-hq/ant/releases

Can I run the pieces myself?

A gateway, an HTTP server that fetches Swarm content for browsers without a node of their own, is the easy way to start, but it means depending on whoever runs it. You can run your own gateway in front of your own node instead. For development, a local Swarm on your laptop needs no real funds, and SwarmScan shows the live network from the outside.

Swarm gatewaysWhat a gateway does, and how to run your own in front of a Bee node.docs.ethswarm.org/docs/develop/tools-and-features/gateway-proxy bee-factoryA local Swarm to develop and test apps against.docs.ethswarm.org/docs/develop/tools-and-features/bee-dev-mode Private test networkA multi-node network you control.docs.ethswarm.org/docs/develop/tools-and-features/starting-a-test-network fdp-playA Docker-based local Bee cluster.github.com/fairDataSociety/fdp-play SwarmScanNetwork explorer and node statistics.swarmscan.io

Swarm, under the hood

How does a request find its data?

What happens when the publisher goes offline?

What if your browser is the node?

Can I run the pieces myself?

Go deeper

This page lives on Swarm