Skip to content

test: make isAttestation handler work with negative tests#9466

Merged
nflaig merged 4 commits into
ChainSafe:unstablefrom
markolazic01:fix/is-attestation-handler
Jun 6, 2026
Merged

test: make isAttestation handler work with negative tests#9466
nflaig merged 4 commits into
ChainSafe:unstablefrom
markolazic01:fix/is-attestation-handler

Conversation

@markolazic01

Copy link
Copy Markdown
Contributor

Motivation

Three Gloas on_attestation spec test vectors added in ethereum/consensus-specs#5275 were skipped in #9422 with a TODO pending investigation. This PR resolves that investigation and re-enables the tests.

Description

The validation logic in validateAttestationData was already correct — all three payload-status checks were implemented and firing as expected. The root cause was that the attestation step handler in fork_choice.test.ts had no valid flag handling: it called onAttestation unconditionally and let any thrown ForkChoiceError propagate as a test failure, even when the spec vector marks the attestation as valid: false.

Fix: wrap onAttestation in the same try/catch + isValid pattern already used by the execution_payload step handler, then unskip the three tests.

Closes #9447

AI Assistance Disclosure

Used Claude as a debugging collaborator to trace the failure from error output to root cause and to draft the descriptions. Code changes authored manually.

@markolazic01 markolazic01 requested a review from a team as a code owner June 5, 2026 12:18

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the fork choice spec tests to handle negative test cases for attestations by wrapping the onAttestation call in a try-catch block. It also re-enables several previously skipped validate_on_attestation tests. Feedback on these changes suggests moving the test setup (getIndexedAttestation) outside of the try-catch block to prevent false positives, and using safe type narrowing (instanceof Error) instead of unsafe type casting.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +188 to +196
const attestation = testcase.attestations.get(step.attestation);
if (!attestation) throw Error(`No attestation ${step.attestation}`);
const headState = chain.getHeadState() as BeaconStateView;
const attDataRootHex = toHexString(sszTypesFor(fork).AttestationData.hashTreeRoot(attestation.data));
chain.forkChoice.onAttestation(
headState.cachedState.epochCtx.getIndexedAttestation(ForkSeq[fork], attestation),
attDataRootHex
);
try {
chain.forkChoice.onAttestation(
headState.cachedState.epochCtx.getIndexedAttestation(ForkSeq[fork], attestation),
attDataRootHex
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There are two improvements we can make here:

  1. Move test setup outside the try block: getIndexedAttestation is part of the test setup/harness. If it throws an error (e.g., due to a malformed state or missing epoch context), that error would be caught by the try/catch block. In a negative test (isValid is false), this setup failure would be incorrectly interpreted as a successful validation failure, leading to a false positive. Moving it outside ensures we only catch errors thrown by onAttestation itself.
  2. Use safe type narrowing: Casting e as Error is unsafe if e is not an instance of Error (or is null/undefined). Using e instanceof Error is the standard and safer way to narrow the unknown error type in TypeScript.
              const indexedAttestation = headState.cachedState.epochCtx.getIndexedAttestation(ForkSeq[fork], attestation);
              try {
                chain.forkChoice.onAttestation(
                  indexedAttestation,
                  attDataRootHex
                );
                if (!isValid) throw Error("Expect error since this is a negative test");
              } catch (e) {
                if (isValid || (e instanceof Error && e.message === "Expect error since this is a negative test")) throw e;
              }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied 1., skipped 2. to follow the pattern from other tests.

@markolazic01

Copy link
Copy Markdown
Contributor Author

Formatted, all checks should pass now.

@nflaig nflaig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @markolazic01 for investigating and fixing this!

@nflaig nflaig changed the title fix: make isAttestation handler work with negative tests test: make isAttestation handler work with negative tests Jun 6, 2026
@nflaig nflaig merged commit 79c77e2 into ChainSafe:unstable Jun 6, 2026
20 checks passed
@nflaig

nflaig commented Jun 6, 2026

Copy link
Copy Markdown
Member

AI Assistance Disclosure

feel free to omit this from your PRs going forward if you like

@markolazic01

Copy link
Copy Markdown
Contributor Author

feel free to omit this from your PRs going forward if you like

Will do, thanks!

@wemeetagain

Copy link
Copy Markdown
Member

🎉 This PR is included in v1.44.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate why validate_on_attestation are failing since alpha.9

3 participants