Skip to content
datastore.sh

Inside the indexing architecture behind our Solana coverage

Engineering · 9 min read · representative content

Why instruction-level capture matters

Most Solana datasets are built from transaction logs, because logs are easy to subscribe to. The problem is that logs are a lossy projection of what actually happened: programs choose what to emit, formats drift between releases, and inner instructions — where most protocol activity actually occurs — are often invisible.

Our capture layer works from the transaction itself: every instruction, every inner instruction, every account delta, at slot granularity. Decoding then becomes a deterministic function of bytes we permanently hold, not of whatever a program happened to log that week.

Program upgrades are the real enemy

A protocol like Pump.fun or Meteora ships program upgrades that change instruction layouts without notice. If your decoder assumes one layout, an upgrade silently corrupts everything you index afterward.

We pin decoders to program deployments. When a program upgrades, the affected instruction stream is quarantined until the new layout is verified, decoded output is compared across the boundary, and only then does the new decoder version enter production. The decoder version that produced every row is recorded, so any decoding question is answerable after the fact.

Reorgs, duplicates, and the boring work

Solana forks. Capture paths overlap. The same instruction can arrive twice through different routes. None of this is exotic — it is the everyday grind of chain infrastructure, and it is exactly where quiet data corruption comes from.

Every record passes through reorg resolution against finalized state and content-hash deduplication before it can enter a published dataset. The reconciliation queue is boring by design: when it is empty, the archive matches the chain.

What this means for the datasets you buy

The practical consequence is that a dataset like Pump.fun Trade History is reproducible: the same raw capture and the same decoder version always produce the same Parquet files, byte for byte. That is what lets us publish checksums, version datasets immutably, and treat corrections as new versions rather than silent rewrites.