Skip to content
datastore.sh

Streams

The catalog, live — delivered to your sink.

Every archive on datastore.sh is produced by pipelines that run continuously. Streams puts those pipelines to work for you: the same decoded tables, flowing into your own Kafka, ClickHouse, NATS, or S3 — priced per pipeline, cancel any month.

Browse streamable datasets Stream pricing

24 datasets streamable today · more in certification

Destinations

Data lands where your access controls already apply

Kafka

One topic per table, protobuf-encoded, schema-registry ready.

needs: bootstrap servers + sasl credentials with write access to a topic prefix

ClickHouse

Rows inserted directly into your tables, DDL generated from our schemas.

needs: host, database, and a user with insert on the target tables

NATS JetStream

One subject per table with deterministic message IDs for dedup.

needs: server url and an auth token or credentials file

S3 (Parquet drops)

New Parquet partitions land in your bucket on a fixed cadence.

needs: bucket, prefix, and keys with putobject on that prefix

what arrives · kafka example

# one Kafka message per decoded row — protobuf, schema-registry ready
topic:   acme.solana.pumpfun.buy
key:     <block_hash>:<tx_index>:<ix_index>      # stable dedup identity
headers: schema=v1.0  block=354211882
payload: { slot, block_time, signature, mint, user, amount, ... }

From checkout to first message

Four steps, one business day at most

01

Pick a dataset

Any streamable dataset in the catalog. The stream carries the same decoded tables and schema versions as the archive.

02

Point it at your sink

Kafka, ClickHouse, NATS JetStream, or S3. Subscribe on the dataset page; connect credentials through a secure link after payment.

03

We verify with a test write

Before the stream starts, the platform performs a live test write with your credentials so misconfiguration surfaces immediately, not silently.

04

Data flows, health is public

Cursor-committed delivery with a per-pipeline status page: current block, head lag, and the full gap and duplicate-risk record.

Delivery guarantees

The archive's integrity machinery, running for you

Delivery is at-least-once with deterministic dedup keys and continuous gap reconciliation — the same integrity machinery that audits our archives.

Cursor-committed resume

Every pipeline commits its position as it delivers. Restarts resume exactly where they stopped — no manual replays, no silent skips.

Gap tracking, not gap hoping

Missing blocks and skipped ranges are detected, recorded, and repaired — the same reconciliation machinery that audits our archives.

Deterministic dedup keys

Delivery is at-least-once with stable per-message identity, so exactly-once lands where it belongs: enforceably, in your sink.

An honest health record

Every pipeline exposes its delivery-health history — including any duplicate-risk windows after failover, with reconciliation status.

Backfill + Tail

History and live, stitched at a single block

Buying the archive and the stream together? The stream starts at the exact block your archive ends — one schema, no gap, no overlap. When the archive is fulfilled we record its final block; your stream's cursor starts there. The seam isn't a promise — it's a number both deliveries share, and your sink can verify it.

Pick a dataset to bundle

the seam · pump_fun example

archive   …/pump_fun/schema=v1.0/
          final block  354,211,881   ← recorded at fulfillment

stream    starts at    354,211,882   ← cursor start
          same schema  v1.0
          overlap      0 blocks
          gap          0 blocks

Streamable today

24 datasets, live from the catalog

Need one that isn't listed? Every dataset page takes stream requests — most certifications ship within two weeks.

Pricing

Per pipeline, per month. That's the whole model.

Starter

$99 / pipeline / month

  • 1 pipeline · 1 destination
  • All tables in the dataset
  • Start from chain head or a chosen block
  • Delivery-health status page

Pro

$299 / pipeline / month

  • 1 pipeline · 1 destination, priority capacity
  • All tables; custom table selection on request
  • Backfill + Tail: start at your archive's final block
  • Named engineer contact for the pipeline
Start on a dataset page Need isolation or custom transforms?