Skip to content

feat(copycat): headers-only indexing mode with fail-closed completion - #7

Open
xylophonez wants to merge 2 commits into
edgefrom
feat/copycat-headers-mode
Open

xylophonez wants to merge 2 commits into
edgefrom
feat/copycat-headers-mode

Conversation

@xylophonez

Copy link
Copy Markdown
Owner

Summary

Adds a headers-only indexing mode to the ~copycat@1.0 Arweave engine (dev_copycat_arweave), plus fail-closed completion semantics for it.

Headers mode replicates a block range using one block2 layout request per block and fetches individual transaction headers only for the zero-data transactions that require them, instead of pulling every transaction. For low-fanout consumers — resource-constrained or edge nodes backfilling a large historical range from the node's native Arweave peer pool — this collapses the request count dramatically (a typical tip block resolves in ~2 requests instead of one-per-transaction) while writing the same canonical header/index entries as the existing path.

Behaviour

  • Opt-in. Existing copycat modes are unchanged; headers mode is selected explicitly. Default behaviour is identical to before.
  • Fail-closed. A malformed or mismatched block2 layout, an out-of-order/incorrect block id, or a header that cannot be retrieved is treated as a hard failure. The engine never returns a success status for a partially-indexed range and never writes a per-block completion marker after a skipped header. A completion marker therefore means "every required header for this block is committed locally," so downstream consumers can trust it as a real guarantee rather than a best-effort hint.
  • No silent remote fallback. Header/layout retrieval stays on the node's configured native routes; a missing local entry after a committed marker is a definite state ("data-bearing, intentionally not held"), not a trigger to reach upstream.

Scope

Single file: src/preloaded/query/dev_copycat_arweave.erl. No changes to core, stores, HTTP, or other devices.

Testing

Covered by the copycat package suite, including an HTTP-level fail-closed test that drives a valid block, a malformed block2, and an invalid backing store through the real packaged HTTP path and asserts a non-2xx result with no completion marker written while still only touching the native block/block2 routes.

@xylophonez
xylophonez force-pushed the feat/copycat-headers-mode branch from 1ea08ad to 831e3e9 Compare August 20, 2026 21:18
@xylophonez
xylophonez force-pushed the feat/copycat-headers-mode branch from 831e3e9 to 4b29d56 Compare August 20, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant