Skip to content

Conversation

@Subhasish-Behera
Copy link

@Subhasish-Behera Subhasish-Behera commented Nov 19, 2025

Implements caching for Payload Timeliness Committee (PTC) to avoid recomputing balance-weighted selection multiple times per epoch.

Problem: PTC uses expensive balance-weighted selection. Without caching, the same committee is recomputed for every
validator duty check and every attestation verification.

Solution: Cache the entire epoch's PTC with constant time validator lookup.

New Functions:

  • get_validator_ptc_slots(validator_index) - Returns all (slot, position) pairs where validator appears.
    will be used in validator duty calls to know which slots to attest.
  • get_ptc(state, slot) - Returns full PTC committee for a slot. Used for gossip verification, block processing.

hangleang and others added 7 commits November 5, 2025 18:27
* Add Gloas helper, state transition functions

* Add DataColumnSidecar as Enum

* Add Gloas attestation gossipsub rules

* Extract slot bytes for data column sidecar, deal with zero bid correctly

* apply suggestions
@Subhasish-Behera Subhasish-Behera marked this pull request as ready for review November 19, 2025 06:09
@Subhasish-Behera Subhasish-Behera changed the title Subh/ptc cache new PTC Cache Implementation Nov 19, 2025
* Add Gloas PTC assignment and payload attestation flow

* Extend `TickKind` to support `PayloadAttestation` for Gloas

* Payload attestation validation

* Add payload attestation aggregation pool

* Prepare payload attestations in block producer
Adds PTC caching for Gloas fork. Stores entire epoch.
Lazy init via OnceCell.
- stores all occurrences for validators with multiple ptc slots.
Changes ptc_positions to Vec<Vec<(Slot, usize)>>.
- higher memory usage.
- Hybrid approach: Vec for single occurrences, HashMap for multiples.
- build with pre-counting.
- Better memory efficiency.
- Balance-weighted selection depends on effective_balance which changes every epoch.
unlike other existing cache object(s).
- Only cache Current epoch, clear on transition.
- Reduces memory usage.
- committee cache keeps multiple epochs (balance dosent affect).
@Subhasish-Behera
Copy link
Author

@hangleang can you have a look at this. i am planning to get these merged to be used in validator duty calls

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you simply explain what it does? what technique you use here to manage the cache? I can't fully wrap my head around

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @hangleang so the goal is to reduce the calls and do the actual calculation once per epoch.

So it operates like a regular cache. But the added element is that the balance_weighted_selection. So that's on top of seed and validator indices.

Call Flow

` get_ptc(state, slot)

state.ptc_cache().get_or_init() ← OnceCell: thread-safe lazy init

build_ptc_cache(state, epoch)

for each slot in epoch:
compute_ptc_for_slot_internal() ← Calls balance_weighted_selection

Store in ptc_shuffling: [slot0: 512 validators, slot1: 512, ..., slot31: 512]

Build reverse indexes for validator→slots lookup `

so in first commit i made 2 mistakes 1. keeping a cache for 3 epochs but thats wrong as balance can change per epoch. 2. didnt handle multiple occurance of single validator for reverse lookup ptc_positions(used in get_validator_ptc_slots which answers which slots is this validator assigned).

in 2nd commits. instead of keeping the track of validators position in a vec i kept in a vec<vec<>> to give multiple positons per validator.

in 3rd commit , instead of creating vec vec for every validator, i changed to maintaining 2 lookups oneptc_positions for single occurance and other multiple_positions for validators with multiple occurances. thus reducing the chances of 2 vector overhead for every single validator index. manged by a hash map with vec of position of slots as value(validator index as key)

in 4th commit, i fixed my mistake of keeping/precalculating ptc for multiple epochs assuming that balance would remain same across multiple epochs.

the whole constraint of build_ptc_cache will be need once per epoch is managed by the cache clearing logic by calling clear_ptc_cache in process_epoch.

  1. Forward index (get_ptc): slot → validators
    - Uses ptc_shuffling (flat Vec)
    - "answers who are the ptc members for slot x"
  2. Reverse index (get_validator_ptc_slots): validator → slots
    - Uses ptc_positions + multiple_positions
    - "validator v is assigned to which slots"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @Subhasish-Behera, I like the idea to cache PTC members for an epoch which can be used in validate_payload_attestation. currently, we do have own_ptc_members.rs which keep track of assigned slots for validators attached to the beacon node (we don't need to keep track other validators assigned slots, which is non of the beacon node business), so I think we don't need the reverse lookup per validator in this cache, this way we can make the cache less complex. I also don't think keep ptc cache within state is the right place, the best place might be in fork_choice_store

@hangleang hangleang force-pushed the epbs branch 2 times, most recently from 968d5ac to 2a96e60 Compare November 28, 2025 03:09
@hangleang hangleang added the epbs label Dec 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants