partialmessages: add per peer bounds on peer initiated groups #650
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MarcoPolo
force-pushed
the
per-peer-bounds-pm
branch
from
c7e1d41 to
4bca56e
Compare
sukunrt
approved these changes
Oct 31, 2025
partialmessages/partialmsgs.go
Outdated
Comment on lines
+120
to
+128
| // PeerInitiatedGroupLimitPerTopic limits the number of Group states all | ||
| // peers can initialize per topic. A group state is initialized by a peer if | ||
| // the peer's message marks the first time we've seen a group id. | ||
| PeerInitiatedGroupLimitPerTopic int | ||
|
|
||
| // PeerInitiatedGroupLimitPerTopicPerPeer limits the number of Group states | ||
| // a single peer can initialize per topic. A group state is initialized by a | ||
| // peer if the peer's message marks the first time we've seen a group id. | ||
| PeerInitiatedGroupLimitPerTopicPerPeer int |
Member
There was a problem hiding this comment.
I'd recommend rephrasing the comments to talk of messages with group IDs as opposed to group state. The state is private to the package so the comment might be confusing. The clarifying statement helps, but if it's directly related to the number of new messages that'd be clearer.
Contributor
Author
There was a problem hiding this comment.
I'm not sure phrasing this in terms of messages is clearer. The thing this is limiting is the number of group ID related state for a group ID we don't know about.
Messages sounds like we are persisting application data, which we aren't.
Do you have a another suggestion?
MarcoPolo
force-pushed
the
per-peer-bounds-pm
branch
from
4bca56e to
230b041
Compare
MarcoPolo
force-pushed
the
supports-partial
branch
from
a306e04 to
a9c155a
Compare
MarcoPolo
force-pushed
the
per-peer-bounds-pm
branch
from
230b041 to
62d3416
Compare
MarcoPolo
force-pushed
the
supports-partial
branch
from
a9c155a to
eedfab5
Compare
MarcoPolo
force-pushed
the
per-peer-bounds-pm
branch
from
62d3416 to
cc78ea4
Compare
A peer initiated group state is a state that is initiatlized due to receiving a partial message RPC for which we have not (yet) published to. It's useful to have this state so we can remember what parts a peer has if we do publish to that group. Previously we limited the total number of peer initiated group states in a topic. This was agnostic to how many groups a single peer initiated, which meant that a single peer could exhaust the whole limit. This change tracks how many group states each peer has initialized and limits each peer to a configurable number of these states. The default total and per peer limits of 255 8 are likely good enough for most use cases, but the options exist for users to change this if they need to.
changes partialMessageStatePerTopicGroup to partialMessageStatePerGroupPerTopic to better reflect its usage.
MarcoPolo
force-pushed
the
supports-partial
branch
from
eedfab5 to
2c63415
Compare
MarcoPolo
force-pushed
the
per-peer-bounds-pm
branch
from
cc78ea4 to
d704f03
Compare
Base automatically changed from
supports-partial
to
marco/partial-messages-2
November 14, 2025 00:19
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A peer initiated group state is a state that is initiatlized due to receiving a partial message RPC for which we have not (yet) published to.
It's useful to have this state so we can remember what parts a peer has if we do publish to that group.
Previously we limited the total number of peer initiated group states in a topic. This was agnostic to how many groups a single peer initiated, which meant that a single peer could exhaust the whole limit.
This change tracks how many group states each peer has initialized and limits each peer to a configurable number of these states. The default total and per peer limits of 255 8 are likely good enough for most use cases, but the options exist for users to change this if they need to.
cc @kasey who suggested the idea.