Skip to content

Bring the docs level with what is built - #28

Merged
Wired4ncer merged 1 commit into
mainfrom
docs/state-of-the-ingest
Aug 9, 2026
Merged

Wired4ncer merged 1 commit into
mainfrom
docs/state-of-the-ingest

Conversation

@Wired4ncer

Copy link
Copy Markdown
Owner

Five pull requests landed since these documents were last true, and they described a project where none of the ingest existed.

The build order says what is built

…and, more usefully, what is built but not yet proven against a live node. The block path and chain catch-up have only ever run against fixtures and a fake chain. A repair path that has never run against a real node is not known to work, so that distinction is written down rather than left implied by an open issue — it is the first thing a reader should be able to find.

What is proven live: the sockets and the transaction parser, against 254 real mainnet transactions, none malformed. Until then the parser had only ever seen its own synthetic fixtures.

New: what the node must provide

Nothing stated this anywhere, which is how a whole class of failure stayed invisible. Two ZMQ endpoints, four RPC methods, a pruned node — plus the two facts that cost the most to learn:

  • The whitelist is not visible to any test. Under rpcwhitelistdefault=0, a method the code calls but the whitelist omits fails in a way no offline test can predict. Re-check the calls against the deployed whitelist whenever either changes.
  • The prune target is the catch-up window. Blocks below the prune height cannot be refetched, so a service down longer than that cannot repair itself from blocks — and fails loudly rather than pretending.

It also records why getblockheader is deliberately not used, because the natural call is the forbidden one and someone will reach for it again. getblock at verbosity 1 carries the same two fields, and one method fewer on the whitelist is the difference between a compromised service being able to read the chain and being able to act on the node.

CONTRIBUTING was missing the install step

pip install -e . appeared nowhere — the tests import coldwatch, so anyone following the documented steps from a clean clone would have failed on the first command. requirements-dev.txt also still described pyzmq as tooling for the stream replay; it is now the runtime's one third-party dependency, so both the step and the reason are stated where a contributor actually hits them.

Adding a second runtime dependency is now explicitly described as a decision that belongs in a pull request arguing for it, rather than something that can arrive by convenience.

Checks

Documentation only — no source changes. 176 tests, ruff clean.

🤖 Generated with Claude Code

Five pull requests landed since these were last true, and the docs described a
project where none of the ingest existed.

The build order now says which steps are built and which are open, with a table
for the ingest core specifically -- and, more usefully, which parts are built
but NOT yet proven against a live node. The block path and chain catch-up have
only ever run against fixtures and a fake chain, and a repair path that has
never run against a real node is not known to work. That distinction is the
first thing a reader should be able to find, so it is written down rather than
implied by an open issue.

A new section states what the node must provide, which nothing did before. Two
ZMQ endpoints, four RPC methods, and a pruned node -- plus the two facts that
cost the most to learn: that the whitelist must be checked against the code's
calls whenever either changes, and that the prune target IS the catch-up window,
so a service down longer than that window cannot repair itself from blocks.

It also records why getblockheader is not used, because the natural call is the
forbidden one and someone will reach for it again. getblock at verbosity 1
carries the same two fields, and one method fewer on the whitelist is the
difference between a compromised service being able to read the chain and being
able to act on the node.

CONTRIBUTING was missing `pip install -e .` entirely and described pyzmq as
tooling for the stream replay. It is now the runtime's one third-party
dependency, so both the install step and the reason are stated where a
contributor will actually hit them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Wired4ncer
Wired4ncer merged commit 819e783 into main Aug 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant