Skip to content

feat: cache the last 2 PayloadEnvelopeInputs#9249

Open
twoeths wants to merge 43 commits intonc/alpha.5-vibefrom
nc/alpha.5-vibe-te/cache-2-payloads
Open

feat: cache the last 2 PayloadEnvelopeInputs#9249
twoeths wants to merge 43 commits intonc/alpha.5-vibefrom
nc/alpha.5-vibe-te/cache-2-payloads

Conversation

@twoeths
Copy link
Copy Markdown
Contributor

@twoeths twoeths commented Apr 21, 2026

Motivation

  • We should not prune PayloadEnvelopeInput from seenPayloadEnvelopeInputCache right after we persist to DB, because block production needs it (next-slot getParentExecutionRequests and prepareExecutionPayload read the cached envelope synchronously).

Description

AI Assistance Disclosure

Used Claude Code to assist with implementation and review.

nflaig and others added 30 commits April 13, 2026 16:21
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
Implement the `should_apply_proposer_boost` logic from consensus-specs
commit 71d1151 (PR #4807). This addresses the builder reveal safety
concern where a colluding next-slot proposer could use proposer boost
to override a legitimately revealed block.

Changes:
- Add `ptcTimeliness` and `proposerIndex` fields to ProtoBlock
- Add `isBlockPtcTimely` to track PTC deadline timeliness
- Add `shouldApplyProposerBoost` which withholds boost when the parent
  is a weak, equivocating block from the previous slot
- Add `findEquivocatingBlocks` in ProtoArray to detect proposer
  equivocations by scanning for PTC-timely blocks at the same slot
  from the same proposer
- Gate proposer boost in `getWeight` on `shouldApplyProposerBoost()`
- Pre-gloas blocks retain unconditional boost (backward compatible)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…cs#5094)

- Add executionRequestsRoot to ExecutionPayloadBid
- Remove stateRoot from ExecutionPayloadEnvelope
- Add parentExecutionRequests to BeaconBlockBody

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…eturn

New function implements deferred parent execution payload processing
(consensus-specs#5094). Moves execution requests, builder payment, and
availability updates from envelope processing to block processing.

Removes the isParentBlockFull early return in processWithdrawals since
processParentExecutionPayload now runs before withdrawals.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lock

New ordering for Gloas blocks:
1. processParentExecutionPayload (NEW - before header)
2. processBlockHeader
3. processWithdrawals (no longer has empty-parent early return)
4. processExecutionPayloadBid
5. ... rest unchanged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Envelope verification no longer mutates state. All state effects (execution
requests, builder payment, availability, latestBlockHash) have moved to
processParentExecutionPayload in the next block.

Changes:
- Remove state cloning, execution request processing, builder payment,
  availability/latestBlockHash updates, and state root verification
- Add executionRequestsRoot check against bid commitment
- Return void instead of post-state
- Remove verifyStateRoot and dontTransferCache options
- Rename stateView method to verifyExecutionPayloadEnvelope

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…onPayload

With deferred payload processing, the envelope no longer produces a
separate post-state. The FULL variant node shares the same stateRoot
as the PENDING node.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- importExecutionPayload now does pure verification only — no state
  cloning, no post-payload state computation, no state root check,
  no regen.processState(), no checkpoint caching
- Remove stateRoot from executionPayload and executionPayloadGossip events
- Remove stateRoot from envelope construction in validator API
- Remove stateRoot from fork choice onExecutionPayload call
- Envelope verification runs via verifyExecutionPayloadEnvelope (void)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rocessing

Block production:
- Add executionRequestsRoot to ExecutionPayloadBid construction
- Add parentExecutionRequests to BeaconBlockBody via getParentExecutionRequests()
- Add getParentExecutionRequests() to BeaconChain (returns parent's
  execution requests from cached envelope, or empty if EMPTY parent)

Gossip validation:
- Add executionRequestsRoot check in envelope validation
- Add EXECUTION_REQUESTS_ROOT_MISMATCH error code

Cleanup:
- Remove stateRoot from validator envelope logging
- Fix spec test for void-returning processExecutionPayloadEnvelope
- Update stale TODO comment in writePayloadEnvelopeInputToDb

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use electra.ExecutionRequests type instead of unknown[]
- Fix parentBlockRootHex -> parentBlock.blockRoot variable name

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Range sync now fetches and validates execution payload envelopes alongside
blocks for Gloas forks. With deferred processing, blocks can sync
optimistically without envelopes, but envelopes are fetched in parallel
for fork-choice FULL/EMPTY variant accuracy.

Changes:
- Batch carries payloadEnvelopes across all state transitions
- downloadByRange fetches envelopes via sendExecutionPayloadEnvelopesByRange
- validateEnvelopesByRangeResponse verifies beaconBlockRoot matches blocks
- processChainSegment accepts payloadEnvelopes parameter
- SyncChainFns types updated for envelope data flow
- Add INVALID_ENVELOPE_BEACON_BLOCK_ROOT error code

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a gossip block claims its parent was FULL but the parent's envelope
hasn't been seen, the block is queued and the parent envelope is fetched
via sendExecutionPayloadEnvelopesByRoot.

Changes:
- Add PARENT_PAYLOAD_UNKNOWN block error code and gossip validation
- Add PendingPayloadEnvelope type for tracking missing envelopes
- BlockInputSync: pendingPayloads map, triggerPayloadSearch,
  fetchPayloadEnvelope, onUnknownPayloadEnvelope handler
- Subscribe to unknownEnvelopeBlockRoot and executionPayloadAvailable events
- Add metrics for payload fetch tracking
- Handle PARENT_PAYLOAD_UNKNOWN in processBlock error recovery

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Handle both Fulu and Gloas column shapes in validateColumnsByRangeResponse.
Fulu columns use signedBlockHeader.message.slot, Gloas columns use slot
directly. Dispatch to validateGloasBlockDataColumnSidecars vs
validateFuluBlockDataColumnSidecars based on fork.

Gloas columns in cacheByRangeResponses are skipped (routed to
PayloadEnvelopeInput by the caller, not to IBlockInput).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ensi321 and others added 13 commits April 19, 2026 18:34
- Only check payload-seen when block is known; unknown blocks are
  handled later by verifyHeadBlockAndTargetRoot which enables the
  API retry path for unknown roots
- Use chain.seenPayloadEnvelope which checks both the gossip cache
  (seenPayloadEnvelopeInputCache) and fork choice FULL variant

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move slot from ExecutionPayloadEnvelope to ExecutionPayload as
slotNumber (uint64) and add slotNumber to PayloadAttributes per
consensus-specs#4840. Updates all verification, gossip validation,
signing, serialization, and SSZ byte extraction accordingly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@twoeths twoeths marked this pull request as ready for review April 21, 2026 16:26
@twoeths twoeths requested a review from a team as a code owner April 21, 2026 16:26
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the management of execution payload envelopes by introducing a database fallback for cache misses in getParentExecutionRequests, which has been updated to be asynchronous. It also implements a slot-based pruning mechanism (pruneBelow) for the seenPayloadEnvelopeInputCache, triggered during slot preparation and finalization. Furthermore, BeaconStateView was updated to process ExecutionRequests instead of full envelopes. A review comment suggests restoring a cache check in the gossip validation path to enable faster duplicate detection before executing expensive state regeneration and signature verification.

Comment on lines 52 to 59
const envelopeBlock = chain.forkChoice.getBlockHex(blockRootHex, PayloadStatus.FULL);
const payloadInput = chain.seenPayloadEnvelopeInputCache.get(blockRootHex);
if (envelopeBlock || payloadInput?.hasPayloadEnvelope()) {
if (envelopeBlock) {
throw new ExecutionPayloadEnvelopeError(GossipAction.IGNORE, {
code: ExecutionPayloadEnvelopeErrorCode.ENVELOPE_ALREADY_KNOWN,
blockRoot: blockRootHex,
slot: payload.slotNumber,
});
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The check against seenPayloadEnvelopeInputCache for duplicate detection was removed. While forkChoice is the authoritative signal, it is only updated at the end of the import process. Checking the cache provides a faster way to IGNORE duplicate gossip messages before proceeding to expensive operations like state regeneration and signature verification.

Suggested change
const envelopeBlock = chain.forkChoice.getBlockHex(blockRootHex, PayloadStatus.FULL);
const payloadInput = chain.seenPayloadEnvelopeInputCache.get(blockRootHex);
if (envelopeBlock || payloadInput?.hasPayloadEnvelope()) {
if (envelopeBlock) {
throw new ExecutionPayloadEnvelopeError(GossipAction.IGNORE, {
code: ExecutionPayloadEnvelopeErrorCode.ENVELOPE_ALREADY_KNOWN,
blockRoot: blockRootHex,
slot: payload.slotNumber,
});
}
const envelopeBlock = chain.forkChoice.getBlockHex(blockRootHex, PayloadStatus.FULL);
const payloadInput = chain.seenPayloadEnvelopeInputCache.get(blockRootHex);
if (envelopeBlock || payloadInput?.hasPayloadEnvelope()) {
throw new ExecutionPayloadEnvelopeError(GossipAction.IGNORE, {
code: ExecutionPayloadEnvelopeErrorCode.ENVELOPE_ALREADY_KNOWN,
blockRoot: blockRootHex,
slot: payload.slotNumber,
});
}

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2a3586e6b2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +101 to +102
if (input.slot < slot) {
this.evictPayloadInput(input);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve pending payload inputs when pruning by slot

pruneBelow() now evicts every entry older than the cutoff, including PayloadEnvelopeInputs that still do not have an envelope. That breaks late-envelope recovery paths: UnknownBlockSync.fetchPayloadEnvelope() and validateExecutionPayloadEnvelope() both require an existing payload input to attach/validate fetched envelopes, so once these entries are pruned, delayed envelopes for still-relevant blocks are ignored and dependent blocks can remain stuck in PARENT_PAYLOAD_UNKNOWN processing.

Useful? React with 👍 / 👎.

}

// Cache miss (e.g., entry pruned in prepareNextSlot). Fall back to DB.
const envelopeFromDb = await this.db.executionPayloadEnvelope.get(fromHex(parentBlockRootHex));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fall back to archived envelope DB before throwing

The cache-miss fallback only reads db.executionPayloadEnvelope (hot DB) and throws if absent, but finalized envelopes are migrated to executionPayloadEnvelopeArchive. In cases where block production needs execution requests from a FULL parent whose envelope has already been archived, this path will throw a false invariant violation even though the data exists; it should mirror getExecutionPayloadEnvelope() and check archive storage too.

Useful? React with 👍 / 👎.

// Steady state: cache holds just 2 entries — head (parent for next-slot production)
// and head.parent (proposer-boost-reorg fallback). Anything older is evicted.
const finalHead = this.chain.forkChoice.getBlockHexDefaultStatus(updatedHeadRoot);
const finalHeadParent = finalHead && this.chain.forkChoice.getBlockHexDefaultStatus(finalHead.parentRoot);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't really the final head, we might still build on headRoot at the start of the slot, the updatedHeadRoot is just our prediction we make here in case we wanna reorg the current head (due to proposer boost reorg) but the head we build on is only final when we call chain.getProposerHead(slot) at t=0 of the proposal slot

@ensi321 ensi321 force-pushed the nc/alpha.5-vibe branch 2 times, most recently from 3b7e592 to 89e60ae Compare April 22, 2026 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants