feat: cache the last 2 PayloadEnvelopeInputs#9249
feat: cache the last 2 PayloadEnvelopeInputs#9249twoeths wants to merge 43 commits intonc/alpha.5-vibefrom
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
Implement the `should_apply_proposer_boost` logic from consensus-specs commit 71d1151 (PR #4807). This addresses the builder reveal safety concern where a colluding next-slot proposer could use proposer boost to override a legitimately revealed block. Changes: - Add `ptcTimeliness` and `proposerIndex` fields to ProtoBlock - Add `isBlockPtcTimely` to track PTC deadline timeliness - Add `shouldApplyProposerBoost` which withholds boost when the parent is a weak, equivocating block from the previous slot - Add `findEquivocatingBlocks` in ProtoArray to detect proposer equivocations by scanning for PTC-timely blocks at the same slot from the same proposer - Gate proposer boost in `getWeight` on `shouldApplyProposerBoost()` - Pre-gloas blocks retain unconditional boost (backward compatible) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…cs#5094) - Add executionRequestsRoot to ExecutionPayloadBid - Remove stateRoot from ExecutionPayloadEnvelope - Add parentExecutionRequests to BeaconBlockBody Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…eturn New function implements deferred parent execution payload processing (consensus-specs#5094). Moves execution requests, builder payment, and availability updates from envelope processing to block processing. Removes the isParentBlockFull early return in processWithdrawals since processParentExecutionPayload now runs before withdrawals. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lock New ordering for Gloas blocks: 1. processParentExecutionPayload (NEW - before header) 2. processBlockHeader 3. processWithdrawals (no longer has empty-parent early return) 4. processExecutionPayloadBid 5. ... rest unchanged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Envelope verification no longer mutates state. All state effects (execution requests, builder payment, availability, latestBlockHash) have moved to processParentExecutionPayload in the next block. Changes: - Remove state cloning, execution request processing, builder payment, availability/latestBlockHash updates, and state root verification - Add executionRequestsRoot check against bid commitment - Return void instead of post-state - Remove verifyStateRoot and dontTransferCache options - Rename stateView method to verifyExecutionPayloadEnvelope Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…onPayload With deferred payload processing, the envelope no longer produces a separate post-state. The FULL variant node shares the same stateRoot as the PENDING node. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- importExecutionPayload now does pure verification only — no state cloning, no post-payload state computation, no state root check, no regen.processState(), no checkpoint caching - Remove stateRoot from executionPayload and executionPayloadGossip events - Remove stateRoot from envelope construction in validator API - Remove stateRoot from fork choice onExecutionPayload call - Envelope verification runs via verifyExecutionPayloadEnvelope (void) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rocessing Block production: - Add executionRequestsRoot to ExecutionPayloadBid construction - Add parentExecutionRequests to BeaconBlockBody via getParentExecutionRequests() - Add getParentExecutionRequests() to BeaconChain (returns parent's execution requests from cached envelope, or empty if EMPTY parent) Gossip validation: - Add executionRequestsRoot check in envelope validation - Add EXECUTION_REQUESTS_ROOT_MISMATCH error code Cleanup: - Remove stateRoot from validator envelope logging - Fix spec test for void-returning processExecutionPayloadEnvelope - Update stale TODO comment in writePayloadEnvelopeInputToDb Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use electra.ExecutionRequests type instead of unknown[] - Fix parentBlockRootHex -> parentBlock.blockRoot variable name Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Range sync now fetches and validates execution payload envelopes alongside blocks for Gloas forks. With deferred processing, blocks can sync optimistically without envelopes, but envelopes are fetched in parallel for fork-choice FULL/EMPTY variant accuracy. Changes: - Batch carries payloadEnvelopes across all state transitions - downloadByRange fetches envelopes via sendExecutionPayloadEnvelopesByRange - validateEnvelopesByRangeResponse verifies beaconBlockRoot matches blocks - processChainSegment accepts payloadEnvelopes parameter - SyncChainFns types updated for envelope data flow - Add INVALID_ENVELOPE_BEACON_BLOCK_ROOT error code Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a gossip block claims its parent was FULL but the parent's envelope hasn't been seen, the block is queued and the parent envelope is fetched via sendExecutionPayloadEnvelopesByRoot. Changes: - Add PARENT_PAYLOAD_UNKNOWN block error code and gossip validation - Add PendingPayloadEnvelope type for tracking missing envelopes - BlockInputSync: pendingPayloads map, triggerPayloadSearch, fetchPayloadEnvelope, onUnknownPayloadEnvelope handler - Subscribe to unknownEnvelopeBlockRoot and executionPayloadAvailable events - Add metrics for payload fetch tracking - Handle PARENT_PAYLOAD_UNKNOWN in processBlock error recovery Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Handle both Fulu and Gloas column shapes in validateColumnsByRangeResponse. Fulu columns use signedBlockHeader.message.slot, Gloas columns use slot directly. Dispatch to validateGloasBlockDataColumnSidecars vs validateFuluBlockDataColumnSidecars based on fork. Gloas columns in cacheByRangeResponses are skipped (routed to PayloadEnvelopeInput by the caller, not to IBlockInput). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Only check payload-seen when block is known; unknown blocks are handled later by verifyHeadBlockAndTargetRoot which enables the API retry path for unknown roots - Use chain.seenPayloadEnvelope which checks both the gossip cache (seenPayloadEnvelopeInputCache) and fork choice FULL variant Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move slot from ExecutionPayloadEnvelope to ExecutionPayload as slotNumber (uint64) and add slotNumber to PayloadAttributes per consensus-specs#4840. Updates all verification, gossip validation, signing, serialization, and SSZ byte extraction accordingly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request refactors the management of execution payload envelopes by introducing a database fallback for cache misses in getParentExecutionRequests, which has been updated to be asynchronous. It also implements a slot-based pruning mechanism (pruneBelow) for the seenPayloadEnvelopeInputCache, triggered during slot preparation and finalization. Furthermore, BeaconStateView was updated to process ExecutionRequests instead of full envelopes. A review comment suggests restoring a cache check in the gossip validation path to enable faster duplicate detection before executing expensive state regeneration and signature verification.
| const envelopeBlock = chain.forkChoice.getBlockHex(blockRootHex, PayloadStatus.FULL); | ||
| const payloadInput = chain.seenPayloadEnvelopeInputCache.get(blockRootHex); | ||
| if (envelopeBlock || payloadInput?.hasPayloadEnvelope()) { | ||
| if (envelopeBlock) { | ||
| throw new ExecutionPayloadEnvelopeError(GossipAction.IGNORE, { | ||
| code: ExecutionPayloadEnvelopeErrorCode.ENVELOPE_ALREADY_KNOWN, | ||
| blockRoot: blockRootHex, | ||
| slot: payload.slotNumber, | ||
| }); | ||
| } |
There was a problem hiding this comment.
The check against seenPayloadEnvelopeInputCache for duplicate detection was removed. While forkChoice is the authoritative signal, it is only updated at the end of the import process. Checking the cache provides a faster way to IGNORE duplicate gossip messages before proceeding to expensive operations like state regeneration and signature verification.
| const envelopeBlock = chain.forkChoice.getBlockHex(blockRootHex, PayloadStatus.FULL); | |
| const payloadInput = chain.seenPayloadEnvelopeInputCache.get(blockRootHex); | |
| if (envelopeBlock || payloadInput?.hasPayloadEnvelope()) { | |
| if (envelopeBlock) { | |
| throw new ExecutionPayloadEnvelopeError(GossipAction.IGNORE, { | |
| code: ExecutionPayloadEnvelopeErrorCode.ENVELOPE_ALREADY_KNOWN, | |
| blockRoot: blockRootHex, | |
| slot: payload.slotNumber, | |
| }); | |
| } | |
| const envelopeBlock = chain.forkChoice.getBlockHex(blockRootHex, PayloadStatus.FULL); | |
| const payloadInput = chain.seenPayloadEnvelopeInputCache.get(blockRootHex); | |
| if (envelopeBlock || payloadInput?.hasPayloadEnvelope()) { | |
| throw new ExecutionPayloadEnvelopeError(GossipAction.IGNORE, { | |
| code: ExecutionPayloadEnvelopeErrorCode.ENVELOPE_ALREADY_KNOWN, | |
| blockRoot: blockRootHex, | |
| slot: payload.slotNumber, | |
| }); | |
| } | |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2a3586e6b2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (input.slot < slot) { | ||
| this.evictPayloadInput(input); |
There was a problem hiding this comment.
Preserve pending payload inputs when pruning by slot
pruneBelow() now evicts every entry older than the cutoff, including PayloadEnvelopeInputs that still do not have an envelope. That breaks late-envelope recovery paths: UnknownBlockSync.fetchPayloadEnvelope() and validateExecutionPayloadEnvelope() both require an existing payload input to attach/validate fetched envelopes, so once these entries are pruned, delayed envelopes for still-relevant blocks are ignored and dependent blocks can remain stuck in PARENT_PAYLOAD_UNKNOWN processing.
Useful? React with 👍 / 👎.
| } | ||
|
|
||
| // Cache miss (e.g., entry pruned in prepareNextSlot). Fall back to DB. | ||
| const envelopeFromDb = await this.db.executionPayloadEnvelope.get(fromHex(parentBlockRootHex)); |
There was a problem hiding this comment.
Fall back to archived envelope DB before throwing
The cache-miss fallback only reads db.executionPayloadEnvelope (hot DB) and throws if absent, but finalized envelopes are migrated to executionPayloadEnvelopeArchive. In cases where block production needs execution requests from a FULL parent whose envelope has already been archived, this path will throw a false invariant violation even though the data exists; it should mirror getExecutionPayloadEnvelope() and check archive storage too.
Useful? React with 👍 / 👎.
| // Steady state: cache holds just 2 entries — head (parent for next-slot production) | ||
| // and head.parent (proposer-boost-reorg fallback). Anything older is evicted. | ||
| const finalHead = this.chain.forkChoice.getBlockHexDefaultStatus(updatedHeadRoot); | ||
| const finalHeadParent = finalHead && this.chain.forkChoice.getBlockHexDefaultStatus(finalHead.parentRoot); |
There was a problem hiding this comment.
this isn't really the final head, we might still build on headRoot at the start of the slot, the updatedHeadRoot is just our prediction we make here in case we wanna reorg the current head (due to proposer boost reorg) but the head we build on is only final when we call chain.getProposerHead(slot) at t=0 of the proposal slot
3b7e592 to
89e60ae
Compare
Motivation
PayloadEnvelopeInputfromseenPayloadEnvelopeInputCacheright after we persist to DB, because block production needs it (next-slotgetParentExecutionRequestsandprepareExecutionPayloadread the cached envelope synchronously).Description
PayloadEnvelopes in memory (head + head.parent), needed to passfinalizedSync.test.tsin feat: gloas range sync #9242 (part of feat: gloas alpha.5 speed run #9219).AI Assistance Disclosure
Used Claude Code to assist with implementation and review.