Subgraphs, Substreams, Firehose and the Token API

Before Horizon, “The Graph” and “subgraphs” were close enough to synonyms that nobody minded. That stopped being true. The protocol now supports several data services, and getting the vocabulary straight saves a great deal of confusion later.

Firehose is the floor, not a product you query.

Firehose is the extraction layer. It instruments a chain node to emit a complete, ordered stream of block data as flat files, rather than making you ask for it block by block over RPC.

You do not query Firehose from an application. It exists so that everything above it can read chain history sequentially, at file-read speed, instead of at request-response speed. When people say a pipeline is “Firehose-based”, that is what they mean: the slow part was replaced.

Substreams is the parallel transform layer.

Substreams reads the Firehose stream and runs modules over it. The modules are compiled to WebAssembly, usually written in Rust, and they compose: one module’s output is another’s input.

Two properties matter, and they follow from the design rather than from marketing:

• Because modules are deterministic functions over a fixed input stream, their outputs are cacheable. Re-running a pipeline with one changed module does not mean re-reading the chain from genesis.
• Because the work decomposes by block range, it parallelises across machines in a way a sequential handler loop does not.

That is why Substreams is fast at backfilling. Be careful with specific speed multiples you see quoted, including in older Academy material: they depend entirely on the chain, the workload and what you are comparing against.

A subgraph is a definition, and also a thing you query.

A subgraph is three files working together: a schema saying what entities exist, a manifest saying which contracts and events to watch, and mappings saying how to turn an event into a row. Deploy it, and indexers build it and serve GraphQL against it.

This is the original service and still the most common one. It suits the case where you want a typed, queryable model of a particular protocol’s state, and you are willing to define that model yourself.

Substreams and subgraphs are not rivals. A subgraph can be powered by Substreams, taking its entity changes from a Substreams module instead of from event handlers.

The Token API is the one you use when you did not want any of this.

Common token questions have the same shape for everybody. What does this address hold. What moved, when. What is it worth. Which NFTs does it own.

Writing a subgraph for that, per chain, is repeated work with no product in it. The Token API answers those questions over a plain REST interface across several chains, built on Substreams underneath. You get an endpoint and a key, and you skip the authoring step entirely.

The trade is the usual one. You get speed of adoption and lose the ability to model anything the API does not already model.

How to choose, briefly.

The developer path has a full decision matrix. The short version:

• The question is about token balances, transfers, prices or NFTs, and you want to ship this week: Token API.
• You need a typed model of one protocol’s own state, queried by an app: subgraph.
• You are moving very large history, or feeding a warehouse or a stream rather than a GraphQL endpoint, or your backfill time is the problem: Substreams.
• You are operating infrastructure and want the raw stream: Firehose.