Content-Addressed Storage
A content-addressed store names every object by a cryptographic hash of its bytes. The name and the content verify each other: a lookup that returns data can be checked against the key it was fetched under, and identical content stored twice occupies one slot. The pattern shows up wherever systems need immutable data to move through untrusted or uncoordinated caches — git’s object store, package-manager stores, archival deduplicators, and peer-to-peer filesystems all instantiate it.
The worked example: git’s object store
At git’s core is “a simple key-value data store” where inserting content returns a unique key for retrieval: git hash-object computes the SHA-1 of a header plus the content and stores the result under .git/objects/<first-2-hex>/<remaining-38>. Blobs hold file contents, trees hold entries pointing at blobs and subtrees by hash, and commits point at a top-level tree plus parent commits — so the history is a Merkle DAG where every internal link carries the hash of what it points at. Mutable names (branches, tags) live outside the objects, as cheap pointers into the graph. 1
The construction gives three properties that later systems copy directly: any party can verify an object against its name without trusting the sender; the object graph’s integrity follows from the links (a corrupted or substituted descendant changes every ancestor’s hash); and version updates only add objects or move pointers. 2
The general pattern
IPFS (Benet, 2014) generalizes git’s layout into “a generalized Merkle DAG” of content-addressed blocks with content-addressed hyperlinks: immutable objects (blocks, lists, commits) are named by multihash (a self-describing hash header so the function can vary and evolve), and mutable naming is pushed to a separate self-certifying layer (IPNS, following SFS’s HostID = hash(public_key || Location) design) where the name itself certifies the publisher. The paper’s framing: content addressing plus linking forms a data structure “upon which one can build versioned file systems, blockchains, and even a Permanent Web.” 3
Caches without coordination
Scion, Anthropic’s 2026 agent-scale version control system, keeps everything content-addressed for the operational payoff: “a piece of data never changes once it’s stored, and any server can cache it without checking with the others.” Its fetch protocol requests missing trees and blobs recursively from a root tree — every request names an immutable object, so any tier of its daemon/cache/server fleet can answer, and the tiers behave like classic hierarchical caches (>99% hit rates per tier) with no long-lived state and no coordination. 4
Properties and costs
- Verification at read. Hashing the payload reproduces the key; a cache, mirror, or peer can be checked at any time.
- Deduplication for free. Identical content converges on one name; copies of the same object are indistinguishable.
- Cache coherence without coordination. Immutability means no invalidation protocol: any replica can serve any request (the basis of Scion’s fleet design).
- Structural integrity through linking. Merkle DAGs extend verification from content to structure — to trust a tree you only need its root hash.
- Mutability lives at the edges. Something must still change over time: git uses branch pointers, IPFS uses a signed name layer, and both keep the mutable part small and auditable.
- Names don’t say what, only what bytes. Human-meaningful lookup needs a directory or index on top (path history in Scion, prefix trees in auditable key directories); and changing the hash function is a migration of every name (git’s SHA-1 to SHA-256 transition is the running example).
Connections
- scion-agent-scale-source-control — the VCS case study whose cache fleet exists because of this property.
- key-transparency — Merkle trees used as auditable maps: same commitment machinery, applied to “which key vouches for whom” instead of “which bytes this name commits to.”
- fhs-package-placement — the Nix store’s content-addressed layout is the packaging-domain instance: store paths are hashes of build inputs, which is what makes profiles composable symlink forests.
- reproducible-builds — bit-reproducible outputs are what let a content-addressed build store work at all.
Sources
- Scott Chacon and Ben Straub 2014 — Git Objects (Pro Git)
- Juan Benet 2014 — IPFS - Content Addressed, Versioned, P2P File System
- 2026 — Redesigning Source Control at Anthropic