test: make isAttestation handler work with negative tests#9466
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the fork choice spec tests to handle negative test cases for attestations by wrapping the onAttestation call in a try-catch block. It also re-enables several previously skipped validate_on_attestation tests. Feedback on these changes suggests moving the test setup (getIndexedAttestation) outside of the try-catch block to prevent false positives, and using safe type narrowing (instanceof Error) instead of unsafe type casting.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| const attestation = testcase.attestations.get(step.attestation); | ||
| if (!attestation) throw Error(`No attestation ${step.attestation}`); | ||
| const headState = chain.getHeadState() as BeaconStateView; | ||
| const attDataRootHex = toHexString(sszTypesFor(fork).AttestationData.hashTreeRoot(attestation.data)); | ||
| chain.forkChoice.onAttestation( | ||
| headState.cachedState.epochCtx.getIndexedAttestation(ForkSeq[fork], attestation), | ||
| attDataRootHex | ||
| ); | ||
| try { | ||
| chain.forkChoice.onAttestation( | ||
| headState.cachedState.epochCtx.getIndexedAttestation(ForkSeq[fork], attestation), | ||
| attDataRootHex | ||
| ); |
There was a problem hiding this comment.
There are two improvements we can make here:
- Move test setup outside the
tryblock:getIndexedAttestationis part of the test setup/harness. If it throws an error (e.g., due to a malformed state or missing epoch context), that error would be caught by thetry/catchblock. In a negative test (isValidisfalse), this setup failure would be incorrectly interpreted as a successful validation failure, leading to a false positive. Moving it outside ensures we only catch errors thrown byonAttestationitself. - Use safe type narrowing: Casting
e as Erroris unsafe ifeis not an instance ofError(or is null/undefined). Usinge instanceof Erroris the standard and safer way to narrow theunknownerror type in TypeScript.
const indexedAttestation = headState.cachedState.epochCtx.getIndexedAttestation(ForkSeq[fork], attestation);
try {
chain.forkChoice.onAttestation(
indexedAttestation,
attDataRootHex
);
if (!isValid) throw Error("Expect error since this is a negative test");
} catch (e) {
if (isValid || (e instanceof Error && e.message === "Expect error since this is a negative test")) throw e;
}There was a problem hiding this comment.
Applied 1., skipped 2. to follow the pattern from other tests.
|
Formatted, all checks should pass now. |
nflaig
left a comment
There was a problem hiding this comment.
LGTM, thanks @markolazic01 for investigating and fixing this!
feel free to omit this from your PRs going forward if you like |
Will do, thanks! |
|
🎉 This PR is included in v1.44.0 🎉 |
Motivation
Three Gloas
on_attestationspec test vectors added in ethereum/consensus-specs#5275 were skipped in #9422 with a TODO pending investigation. This PR resolves that investigation and re-enables the tests.Description
The validation logic in
validateAttestationDatawas already correct — all three payload-status checks were implemented and firing as expected. The root cause was that the attestation step handler infork_choice.test.tshad novalidflag handling: it calledonAttestationunconditionally and let any thrownForkChoiceErrorpropagate as a test failure, even when the spec vector marks the attestation asvalid: false.Fix: wrap
onAttestationin the sametry/catch+isValidpattern already used by theexecution_payloadstep handler, then unskip the three tests.Closes #9447
AI Assistance Disclosure
Used Claude as a debugging collaborator to trace the failure from error output to root cause and to draft the descriptions. Code changes authored manually.