Skip to main content
The corpus holds rare shapes one entry at a time. The scan describes the other side: which shapes real pubnet traffic contains, and how often each occurs, so an indexer operator can see what the indexer will meet. It read every pubnet ledger from 2 to 65,505,999 from the public data lake, checked each one, and reduced every transaction to a set of shape tokens. The aggregate output is committed in evidence/mainnet-scan/ with a SHA256SUMS file. Every number on this page is from that folder’s README.md and files unless another source is named.

What was read

By LedgerCloseMeta arm (census-facts.tsv, summed per arm):

How the scan ran

Every step is a subcommand of xdrcheck-corpus scan; the full command list is in evidence/mainnet-scan/README.md. The whole-run steps stream unit by unit with bounded memory.

What was checked on every ledger

scan run refuses a ledger, recording why, when any of these fails:
  1. the zstd frame of the lake object;
  2. the decode of the LedgerCloseMetaBatch and its byte-exact re-encode;
  3. the walk of every byte against the pinned XDR definitions (stellar-xdr v30.0);
  4. the header’s stored hash against the SHA-256 of the LedgerHeader;
  5. the join of every transaction result to an envelope, hashed under the pubnet passphrase.
It also records per ledger whether the header’s txSetHash and txSetResultHash match the transaction set and results the ledger carries, whether the object’s metadata (protocol-version, start-ledger) matches the header, and whether the object’s MD5 equals its ETag. scan chain-check then reads every ledger in order and checks that each header’s previousLedgerHash equals the hash of the ledger before it, across unit boundaries. The chain is open at its first end: ledger 2’s previous hash names ledger 1, which is not in the run. What this proves. The lake as served on 2026-11-16 holds every pubnet ledger in the range exactly once, each one decodes and re-encodes byte for byte with rs-stellar-xdr against the v30.0 definitions, and the ledgers form one unbroken hash chain whose transaction sets and results match their headers.

The shape grammar xcc.shape/v1

Implemented in crates/xdrcheck-corpus/src/cmd/scan/shape.rs, rules in crates/xdrcheck-corpus/data/scan-rules-v1.toml. Components. Each transaction is cut into parts: E the envelope without its operations, A its authorization entries, O one operation with its result and its meta, R the result without the operation results, F the fee processing, M the transaction meta frame. Each ledger adds one L component (header, upgrades, transaction set structure). Tokens. Each component is walked against the definitions into a set of tokens, each one string <kind>|<class>|<body>: Two examples:
No token holds bytes taken from the ledger (keys, addresses, amounts, contract IDs), apart from two closed vocabularies of event names and data-map keys. Levels. A level is a filter on the same tokens: G0 keeps T; G1 adds A O E; G2 adds L V S K X; G3 adds D R. A component shape is the hash of its sorted token set; a whole-transaction shape is the hash of its E, R, F, M (and A) shapes, the set of its distinct O shapes with a repeat class, and an operation-count class. Operation order is not part of it. Detail: Shapes, tokens and levels.

What the census found

At G3 (census-summary.tsv, column distinct; token-classes.tsv; cover-summary.tsv): The first count measures the parts transactions are made of; the second counts distinct combinations of those parts.

The mainnet tier: 6,158 transactions that hold every transaction-level token

corpus/mainnet/ lists 6,158 pubnet transactions from 6,150 ledgers. Together they hold all 15,186 transaction-level tokens. Their bytes are not committed; each line locates one transaction in its lake object. How it was selected. scan cover-stream computes a greedy cover: starting from the rarest token, take a transaction that carries it, add every token that transaction holds, repeat until every token is held (cover-summary.tsv, row TOK rare: 6,158 transactions in 6,150 ledgers). xdrcheck-corpus mainnet build then walks every selected object again with the scan’s own code and writes nothing unless each row’s count of new tokens and rarest occurrence equal the cover’s. One line of corpus/mainnet/tier.jsonl: ledger, tx_index (apply order), tx_hash, protocol, lcm_version; the lake key, the object’s sha256 and size; blob_offset of the LedgerCloseMeta in the decompressed object; parts (envelope, result pair, meta, each fee change: XDR type, JSON path, byte offset, length, sha256); tokens (the token IDs this line adds); rarity (an upper bound on how many transactions in the range carry the rarest token; 1 means one transaction). What make verify-mainnet re-checks, from the cache only (make mainnet-fetch fills it, about 1 GB):
  1. each object’s sha256 and size;
  2. decompression, the batch split and the byte-exact re-encode of the ledger;
  3. that the ledger starts at blob_offset;
  4. each part’s sha256, and its byte-for-byte round trip through rs-stellar-xdr as its XDR type;
  5. that cutting the transaction again gives exactly the line’s parts;
  6. the transaction hash under the pubnet passphrase, the header protocol and the LedgerCloseMeta arm;
  7. a fresh token walk of every ledger: each line’s tokens are exactly those no earlier line holds, and the union is 15,186.
A fetch that receives other bytes than the line names keeps them as <key>.drift and fails. What it gives you. 6,158 real transactions, about 1 GB of lake objects, that exercise every transaction-level token pubnet produced in the range. Run them through your indexer with xdrcheck replay --tx or a service rig, and every arm, optional state, length class, entry-change pattern and event signature mainnet has carried reaches your code at least once.

How the scan feeds the rest of the kit

What the scan does not claim

  • Pipelines were not replayed over all of mainnet. The scan computes shapes, tokens and the checks above. It does not run wallet-backend, stellar-etl, Horizon or any SDK.
  • Shapes are those of the lake as served today. The lake’s older ledgers were rewritten by a recent stellar-core: 58,552,310 of the scanned ledgers carry the core-version metadata of stellar-core 23.0.1 (chain-groups.tsv, column core). A meta arm that only older stellar-core versions wrote is absent from the census even where the network once produced it.
  • The tier covers tokens, not combinations. It holds every token, not every whole-transaction shape, and not the 698 tokens that occur only in the ledger-level component L.
  • No CAP tags on tokens. A reliable map from XDR types to CAPs does not exist yet, so the tier does not claim one.
  • The per-ledger output is not in the repository. It is tens of gigabytes. The token table, the covers and the locate index are kept outside it; the committed files are the aggregates, and the commands to rebuild them.
Related: Mainnet scan results, Shapes, tokens and levels.