NODE · LON-01|LONDON --:--:--
DAC | Digital Asset Claims
Technology · Data Pipeline

Digital Asset Claims On Chain Data Pipeline

How raw ledger records are ingested, normalised and reconciled

Before any analysis begins, ledger data is pulled into a normalised structure so that transactions across different chains can be compared on the same terms.

Streaming ledger blocks resolving into normalised transaction rows

Stages Of Ingestion And Normalisation

Collection

Blocks, transactions, receipts and contract events are collected for the chains and periods relevant to the instruction.

Normalisation

Values, decimals, timestamps and address formats are converted to a single internal representation with the original preserved alongside.

Reconciliation

Each imported record is checked against a second independent source, and any divergence is recorded rather than averaged away.

Handling Reorganisations And Gaps

Chain reorganisations, failed transactions and pending states are retained with their status so that a transaction that never settled is not read as a transfer that occurred.

Where a range of blocks could not be retrieved, the gap is recorded as a gap and carried into the report rather than closed by inference.

Limitations

The pipeline can only reflect what a chain publishes. Off-chain movement inside a custodial service leaves no ledger record and is treated as unobserved unless a document supplies it.

Related Pages