Skip to content

fix: defer using proposer duties v2 to gloas #9519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nflaig merged 1 commit into from
Jun 17, 2026
+70 −35
Merged

fix: defer using proposer duties v2 to gloas #9519

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 31 additions & 25 deletions packages/validator/src/services/blockDuties.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import {ApiClient, routes} from "@lodestar/api";
import {ChainForkConfig} from "@lodestar/config";
import {isForkPostFulu} from "@lodestar/params";
import {isForkPostGloas} from "@lodestar/params";
Comment thread
nflaig marked this conversation as resolved.
import {computeEpochAtSlot, computeStartSlotAtEpoch} from "@lodestar/state-transition";
import {BLSPubkey, Epoch, RootHex, Slot} from "@lodestar/types";
import {sleep, toPubkeyHex} from "@lodestar/utils";
Expand All @@ -11,8 +11,11 @@ import {ChainHeaderTracker, HeadEventData} from "./chainHeaderTracker.js";
import {ValidatorStore} from "./validatorStore.js";

/**
* Pre-Fulu only: poll next-epoch proposer duties ~1s before the boundary. Post-Fulu the 1-epoch
* deterministic lookahead lets us pre-fetch via `runEveryEpoch`, so this fast path is unused.
* Pre-Gloas: poll next-epoch proposer duties ~1s before the boundary. The deterministic 1-epoch
* lookahead (EIP-7917) is available post-Fulu, but consuming it requires `getProposerDutiesV2`,
* which was added to beacon-APIs only after Fulu shipped. To avoid depending on an endpoint not
* all clients implement yet, the VC keeps using v1 + this boundary poll until Gloas; only then
* does it switch to the lookahead-based pre-fetch in `runEveryEpoch`.
*
* Historical context: starting Jul 2023 we poll 1s before the next epoch because
* `PrepareNextSlotScheduler` (BN-side) usually finishes the upcoming-epoch transition in ~3s,
Expand Down Expand Up @@ -123,9 +126,10 @@ export class BlockDutiesService {
}

/**
* Baseline per-epoch fetch. Fires at epoch boundaries (and once at startup). Post-Fulu the
* deterministic 1-epoch lookahead lets us also pre-fetch `epoch + 1`; pre-Fulu the next
* epoch's dep_root only stabilizes at the boundary and is handled by `runEverySlotTask`.
* Baseline per-epoch fetch. Fires at epoch boundaries (and once at startup). Post-Gloas the
* deterministic 1-epoch lookahead (consumed via v2) lets us also pre-fetch `epoch + 1`;
* pre-Gloas the next epoch's dep_root only stabilizes at the boundary and is handled by
* `runEverySlotTask`.
*
* Mid-epoch refreshes (e.g. reorgs) are driven by `onNewHead` instead of polling every slot.
*/
Expand All @@ -144,7 +148,7 @@ export class BlockDutiesService {
await this.pollBeaconProposers(epoch);

const nextEpoch = epoch + 1;
if (isForkPostFulu(this.config.getForkName(computeStartSlotAtEpoch(nextEpoch)))) {
if (isForkPostGloas(this.config.getForkName(computeStartSlotAtEpoch(nextEpoch)))) {
await this.pollBeaconProposers(nextEpoch);
Comment on lines +151 to 152

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid v2 prefetch before the Gloas boundary

When the current epoch is the final Fulu epoch, nextEpoch already resolves to ForkName.gloas, so this branch calls pollBeaconProposers(nextEpoch) immediately and that method selects getProposerDutiesV2 for the requested epoch. That still hits /eth/v2/validator/duties/proposer before the chain has reached Gloas—the compatibility case this change is trying to avoid—and if the connected BN does not serve v2 yet the prefetch fails; the last-slot boundary poll is also skipped because slot + 1 is Gloas, leaving first-Gloas-slot proposer duties to a boundary-time race.

Useful? React with 👍 / 👎.

@nflaig nflaig Jun 17, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's fine, at this point every bn should have proposer duties v2 implemented as node operators have to upgrade before the fork happens, ideally multiple epochs before

}
} catch (e) {
Expand All @@ -156,7 +160,7 @@ export class BlockDutiesService {

/**
* Slot-tick handler. Notifies block production for cached proposers in this slot, and on
* the last slot of a pre-Fulu epoch schedules the boundary fetch for `nextEpoch` duties.
* the last slot of a pre-Gloas epoch schedules the boundary fetch for `nextEpoch` duties.
* Reorg detection is handled by `onNewHead`, so this task does not re-poll on every slot.
*/
private runEverySlotTask = async (slot: Slot, signal: AbortSignal): Promise<void> => {
Expand All @@ -169,10 +173,10 @@ export class BlockDutiesService {

const nextEpoch = computeEpochAtSlot(slot) + 1;
const isLastSlotOfEpoch = computeStartSlotAtEpoch(nextEpoch) === slot + 1;
if (isLastSlotOfEpoch && !isForkPostFulu(this.config.getForkName(slot + 1))) {
// Pre-Fulu: 0-epoch proposer lookahead, so the next-epoch dep_root only becomes stable
// as the last block of the current epoch lands. Sleep until ~1s before the boundary
// then fetch — same timing as before this refactor.
if (isLastSlotOfEpoch && !isForkPostGloas(this.config.getForkName(slot + 1))) {
// Pre-Gloas the VC uses v1 and does not pre-fetch the next epoch on the epoch tick, so
// fetch it ~1s before the boundary instead. Pre-Fulu this is required (0-epoch lookahead);
// for Fulu the lookahead exists but the v2 path is deferred to Gloas (see top of file).
this.pollBeaconProposersBeforeBoundary(slot, nextEpoch, signal).catch((e) => {
this.logger.error("Error on pollBeaconProposersBeforeBoundary", {nextEpoch}, e);
});
Expand All @@ -184,14 +188,15 @@ export class BlockDutiesService {

/**
* SSE head-event handler. The beacon-API `head` event carries attester-duty dep_roots,
* which coincide with the proposer dep_roots at a fork-dependent offset:
* which coincide with the proposer dep_roots the VC caches. The offset depends on the
* proposer-duties endpoint the VC uses, which is gated on Gloas (v1 before, v2 after):
*
* Pre-Fulu (proposer dep_root(E) = block@startSlot(E) - 1):
* Pre-Gloas (v1 proposer dep_root(E) = block@startSlot(E) - 1):
* currentDutyDependentRoot ≡ proposer_dep_root(currentEpoch)
* (next-epoch proposer dep_root is not exposed; pre-Fulu falls back to the
* (next-epoch proposer dep_root is not exposed; pre-Gloas falls back to the
* `runEverySlotTask` boundary poll.)
*
* Post-Fulu (proposer dep_root(E) = block@startSlot(E - 1) - 1, EIP-7917):
* Post-Gloas (v2 proposer dep_root(E) = block@startSlot(E - 1) - 1, EIP-7917):
* previousDutyDependentRoot ≡ proposer_dep_root(currentEpoch)
* currentDutyDependentRoot ≡ proposer_dep_root(nextEpoch)
*
Expand All @@ -204,9 +209,9 @@ export class BlockDutiesService {
currentDutyDependentRoot,
}: HeadEventData): Promise<void> => {
const currentEpoch = computeEpochAtSlot(slot);
const isPostFulu = isForkPostFulu(this.config.getForkName(slot));
const isPostGloas = isForkPostGloas(this.config.getForkName(slot));

if (isPostFulu) {
if (isPostGloas) {
await this.refetchIfDepRootChanged(currentEpoch, previousDutyDependentRoot);
await this.refetchIfDepRootChanged(currentEpoch + 1, currentDutyDependentRoot);
} else {
Expand All @@ -229,11 +234,11 @@ export class BlockDutiesService {
}

/**
* Pre-Fulu boundary fetch. Because pre-Fulu proposer shuffling has 0-epoch look-ahead, a
* proposal for the first slot of the new epoch otherwise goes through the slow path every
* time: the proposal can only happen *after* we download and process the new duties from
* the BN. Polling ~1s before the boundary, while `PrepareNextSlotScheduler` is finishing
* the upcoming-epoch transition, lets us land on a hot BN cache and avoid the miss.
* Pre-Gloas boundary fetch. Without a pre-fetched next epoch, a proposal for the first slot
* of the new epoch otherwise goes through the slow path every time: the proposal can only
* happen *after* we download and process the new duties from the BN. Polling ~1s before the
* boundary, while `PrepareNextSlotScheduler` is finishing the upcoming-epoch transition, lets
* us land on a hot BN cache and avoid the miss.
*
* See https://github.com/ChainSafe/lodestar/issues/5792.
*/
Expand Down Expand Up @@ -303,8 +308,9 @@ export class BlockDutiesService {
return;
}

// Post-Fulu the proposer dependent root changed (deterministic proposer lookahead)
const res = isForkPostFulu(this.config.getForkName(computeStartSlotAtEpoch(epoch)))
// Use v2 only post-Gloas: it exposes the post-Fulu deterministic proposer dependent root,
// but was added to beacon-APIs after Fulu shipped, so we defer to it until Gloas (see top).
const res = isForkPostGloas(this.config.getForkName(computeStartSlotAtEpoch(epoch)))
? await this.api.validator.getProposerDutiesV2({epoch})
: await this.api.validator.getProposerDuties({epoch});
const proposerDuties = res.value();
Expand Down
49 changes: 39 additions & 10 deletions packages/validator/test/unit/services/blockDuties.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ vi.mock("../../../src/services/chainHeaderTracker.js");
describe("BlockDutiesService", () => {
const api = getApiClientStub();
const preFuluConfig = createChainForkConfig(defaultConfig);
const postFuluConfig = getConfig(ForkName.fulu);
const fuluConfig = getConfig(ForkName.fulu);
const postGloasConfig = getConfig(ForkName.gloas);
let validatorStore: ValidatorStore;
let pubkeys: Uint8Array[]; // Initialize pubkeys in before() so bls is already initialized

Expand Down Expand Up @@ -114,7 +115,7 @@ describe("BlockDutiesService", () => {
expect(notifyBlockProductionFn).toHaveBeenCalledWith(slot, [pubkeys[0]]);
});

it("Post-Fulu epoch tick fetches current and next epoch proposer duties", async () => {
it("Post-Gloas epoch tick fetches current and next epoch proposer duties", async () => {
const dutiesEpoch0: routes.validator.ProposerDutyList = [{slot: 0, validatorIndex: 0, pubkey: pubkeys[0]}];
const dutiesEpoch1: routes.validator.ProposerDutyList = [{slot: 32, validatorIndex: 1, pubkey: pubkeys[1]}];
const depRootEpoch0 = ZERO_HASH_HEX;
Expand All @@ -128,7 +129,7 @@ describe("BlockDutiesService", () => {

const clock = new ClockMock();
const dutiesService = new BlockDutiesService(
postFuluConfig,
postGloasConfig,
loggerVc,
api,
clock,
Expand Down Expand Up @@ -233,7 +234,7 @@ describe("BlockDutiesService", () => {
expect(api.validator.getProposerDuties).toHaveBeenCalledTimes(1);
});

it("Post-Fulu head event detects dep_root change for both current and next epoch", async () => {
it("Post-Gloas head event detects dep_root change for both current and next epoch", async () => {
const depRootEpoch0 = toHex(Buffer.alloc(32, 10));
const depRootEpoch1 = toHex(Buffer.alloc(32, 11));
const depRootEpoch0Reorg = toHex(Buffer.alloc(32, 20));
Expand All @@ -249,7 +250,7 @@ describe("BlockDutiesService", () => {
);

const clock = new ClockMock();
new BlockDutiesService(postFuluConfig, loggerVc, api, clock, validatorStore, chainHeaderTracker, null);
new BlockDutiesService(postGloasConfig, loggerVc, api, clock, validatorStore, chainHeaderTracker, null);

await clock.tickEpochFns(0, controller.signal);
expect(api.validator.getProposerDutiesV2).toHaveBeenCalledTimes(2);
Expand All @@ -260,8 +261,8 @@ describe("BlockDutiesService", () => {
await onNewHeadCallback({
slot: 0,
head: ZERO_HASH_HEX,
previousDutyDependentRoot: depRootEpoch0Reorg, // post-Fulu maps to proposer_dep_root(currentEpoch)
currentDutyDependentRoot: depRootEpoch1Reorg, // post-Fulu maps to proposer_dep_root(nextEpoch)
previousDutyDependentRoot: depRootEpoch0Reorg, // post-Gloas (v2) maps to proposer_dep_root(currentEpoch)
currentDutyDependentRoot: depRootEpoch1Reorg, // post-Gloas (v2) maps to proposer_dep_root(nextEpoch)
});

expect(api.validator.getProposerDutiesV2).toHaveBeenCalledTimes(4);
Expand Down Expand Up @@ -316,24 +317,52 @@ describe("BlockDutiesService", () => {
expect(api.validator.getProposerDuties).toHaveBeenCalledTimes(1);
});

it("Post-Fulu last slot of epoch does NOT schedule a pre-Fulu boundary fetch", async () => {
it("Post-Gloas last slot of epoch does NOT schedule a boundary fetch", async () => {
api.validator.getProposerDutiesV2.mockResolvedValue(
mockApiResponse({data: [], meta: {dependentRoot: ZERO_HASH_HEX, executionOptimistic: false}})
);

const clock = new ClockMock();
new BlockDutiesService(postFuluConfig, loggerVc, api, clock, validatorStore, chainHeaderTracker, null);
new BlockDutiesService(postGloasConfig, loggerVc, api, clock, validatorStore, chainHeaderTracker, null);

await clock.tickEpochFns(0, controller.signal);
const callsAfterEpochTick = api.validator.getProposerDutiesV2.mock.calls.length;

await clock.tickSlotFns(31, controller.signal);
await new Promise((resolve) => setTimeout(resolve, 10));

// No extra fetch beyond what `runEveryEpoch` already did — boundary poll is pre-Fulu only
// No extra fetch beyond what `runEveryEpoch` already did, the boundary poll is pre-Gloas only
expect(api.validator.getProposerDutiesV2.mock.calls.length).toBe(callsAfterEpochTick);
});

it("Fulu (pre-Gloas) uses the v1 endpoint and the boundary-poll path, not v2", async () => {
const epoch0Duties: routes.validator.ProposerDutyList = [];
const epoch1Duties: routes.validator.ProposerDutyList = [{slot: 32, validatorIndex: 0, pubkey: pubkeys[0]}];

api.validator.getProposerDuties.mockImplementation(async ({epoch}) =>
mockApiResponse({
data: epoch === 0 ? epoch0Duties : epoch1Duties,
meta: {dependentRoot: ZERO_HASH_HEX, executionOptimistic: false},
})
);

const clock = new ClockMock();
new BlockDutiesService(fuluConfig, loggerVc, api, clock, validatorStore, chainHeaderTracker, null);

// Epoch tick fetches only the current epoch via v1 (no v2, no next-epoch pre-fetch)
await clock.tickEpochFns(0, controller.signal);
expect(api.validator.getProposerDutiesV2).not.toHaveBeenCalled();
expect(api.validator.getProposerDuties).toHaveBeenCalledTimes(1);
expect(api.validator.getProposerDuties.mock.calls[0][0]).toEqual({epoch: 0});

// Last slot of the epoch schedules the boundary fetch for nextEpoch, still via v1
await clock.tickSlotFns(31, controller.signal);
await new Promise((resolve) => setTimeout(resolve, 10));
expect(api.validator.getProposerDutiesV2).not.toHaveBeenCalled();
expect(api.validator.getProposerDuties).toHaveBeenCalledTimes(2);
expect(api.validator.getProposerDuties.mock.calls[1][0]).toEqual({epoch: 1});
});

it("Should remove signer from duty across epochs", async () => {
const duties: routes.validator.ProposerDutyList = [
{slot: 0, validatorIndex: 0, pubkey: pubkeys[0]},
Expand Down
Loading