Substreams

Substreams is a different way of getting the same chain data out, aimed at a different constraint. Subgraphs optimise for a typed, queryable model of a protocol. Substreams optimises for throughput and composition.

The model.

You write modules, usually in Rust, compiled to WebAssembly. Each is a deterministic function. They compose into a graph, where one module’s output feeds another’s input.

Two kinds:

• map modules transform input to output for a block. Stateless.
• store modules accumulate state across blocks. Stateful, with defined merge behaviour so parallel processing stays correct.

The input at the root is the Firehose stream: the chain as an ordered sequence of flat files rather than a sequence of RPC calls.

Why this shape is fast.

Two properties, both consequences of the design rather than of optimisation work.

Outputs are cacheable. A module is a deterministic function of a fixed input stream, so its output for a block range can be computed once and reused. Change one module downstream and the upstream work does not have to be redone.

Work parallelises by range. Because the input is a file stream rather than a stateful cursor, block ranges can be processed on different machines simultaneously. Store modules define how partial states merge, which is what keeps that correct.

Compare against a subgraph’s handler loop, which walks blocks in order on one machine by construction.

Where the output goes.

This is the part that most changes how you should think about it. A subgraph’s output goes one place: entities served over GraphQL. Substreams output goes wherever you sink it.

• Into a subgraph, as entity changes. This is the Substreams-powered subgraph pattern, and it means you can keep the GraphQL API while changing what fills it.
• Into a database you operate: Postgres, ClickHouse, a warehouse.
• Into a stream, for something consuming continuously.
• Into files, for analysis.

So Substreams is not a competitor to subgraphs so much as a different layer. You can use it and still serve GraphQL.

When to reach for it.

• Backfill time is your problem. History is large and a handler loop takes days.
• The destination is not GraphQL. You want a warehouse, a stream, or your own database.
• You are computing rather than recording. Aggregations and derived metrics across large ranges, rather than a straightforward event-to-entity mapping.
• You want composition. Building on modules someone else wrote instead of reimplementing decoding.

When not to.

Honestly, more often than the above.

• You want a typed API for one protocol’s state. A subgraph is the shorter path and stays easier to maintain.
• Your history is small. The parallelism is solving a problem you do not have.
• Your team does not write Rust. This is a real cost, not a snobbery point. A subgraph in AssemblyScript that your team can maintain beats a Substreams module that one person understands.
• You want the least moving parts. Substreams introduces a pipeline and a sink you are responsible for.