Forward compatible consensus data structures (EIP-7688)

[etan-status]

EIP-4788 exposed the beacon root to smart contracts, but smart contracts need to be redeployed / upgraded whenever generalized indices change during a fork, even if that fork does not touch any used functionality. That is analogous to an OS without ABI stability, requiring programs to be maintained and re-compiled due to random breakages in OS updates.

This issue expands further to bridges on other blockchains, and also into wallets / dApps that verify data from the beacon chain instead. Such projects do not typically share Ethereum's release cadence.

• https://eips.ethereum.org/EIPS/eip-4788

EIP-7688 introduces forward compatibility for beacon chain structures. Generalized indices remain same when list capacities evolve over forks, containers no longer get re-indexed when reaching a new power-of-2 number of fields, and fields can be deprecated, leaving a gap in the Merkle tree instead of triggering re-indexing.

• https://eips.ethereum.org/EIPS/eip-7688

EIP-7688 was requested for inclusion by popular projects:

• For Electra by Rocketpool: https://xcancel.com/KaneWallmann/status/1816729724145795258
• For Fulu by Lido: Consensus-layer Call 153 pm#1356 (comment)

[paulmillr]

EIP-7688 is now live in https://github.com/paulmillr/micro-eth-signer

The test vectors from the proton drive link above are available in git repo https://github.com/paulmillr/eth-vectors

[etan-status]

@jtraglia Fallout, to be addressed once this PR gets merged:

1. BUILDER_INDEX_FLAG should be changed from 2^40 to 2^63, as 2^40 no longer has any meaning (the state limit is gone). have checked with @wemeetagain, it shouldn't be on a critical path for typescript
   • Change BUILDER_INDEX_FLAG to 2^63 #5402
2. the genesis state builder from the pandaops has to be addressed to create proper roots when gloas_fork_epoch = 0. interim, have to set gloas_fork_epoch = 1 so that genesis uses fulu or earlier. @pk910
   • addressed below
3. the deposit requests serialization limit has to be raised to support 200M gas, the PR here does not affect network limits, it moves existing limits from the type definition to gossip + state transition rules + preset values, it doesn't change them.
4. If PTC_SIZE can evolve over time, the bitvector inside PayloadAttestation should become ProgressiveBitList, and the list inside IndexedPayloadAttestation should become ProgressiveList (but the indexed payload attestation probably shouldn't even be an SSZ type to begin with, it's just an internal helper object).
   • Checked with @potuz "I suspect it'll stay as this forever"

As this PR affects many files, would recommend a 3-alpha plan to give client teams the opportunity to test this one in isolation. (1) a version that includes everything up to this PR, (2) a version with this PR on top, (3) a version with the fallout addressed + other fixes. would be a nice to have the intermediate tests for (1) and (2), if that's not too cumbersome infra-wise, even though only (3) would ship on devnet-7.

[pk910]

tooling side is ready for eip7688. Just needs a dependency bump for the affected tools :)
I've created a branch for:

• genesis generator: glamsterdam-devnet-7 ethpandaops/ethereum-genesis-generator#300
• dora: glamsterdam-devnet-7 ethpandaops/dora#770

kt config snippets to use these branches:

ethereum_genesis_generator_params:
  image: "ethpandaops/ethereum-genesis-generator:glamsterdam-devnet-7"
dora_params:
  image: ethpandaops/dora:glamsterdam-devnet-7