Partial Messages Extension #631

MarcoPolo · 2025-07-15T01:31:43Z

Implements libp2p/specs#685

Commits are meaningful. Best to review commit by commit.

extensions.go

partialmessages/partialmsgs.go

MarcoPolo · 2025-09-24T17:53:49Z

thanks for the review @ppopth, friendly reminder you can see the latest changes with git range-diff master 7fc7eb495bd6 578e12f4afcd

gossipsub_test.go

partialmessages/partialmsgs.go

gossipsub.go

The bitmap is probably the common case, so add library support for it.

This allows a peer's local partsMetadata to merge with their remote partsMetadata. As long as the underlying PartsMetadata type supports CRDT-like addition, we can avoid sending the peer duplicates. Take the following example (credit Sukun): > Say: We have parts 1, 2, 5, 6 > > 1. Peer sends a message saying I have part 1, 2 > 2. We send parts 5, 6 and update the peerstate in PublishMessage to (1, 2, 5, 6) > 3. Concurrently with 2, we receive message from the peer saying I now have parts (1, 2, 3, 4) With this change we can now correctly store the fact that this peer should have parts 1,2,3,4,5,6. And in a way that does not assume a specific representation of the PartsMetadata.

gossipsub.go

pubsub.go

and fix a bug where we would skip sending them full messages if they requested partial messages and we disabled partial messages.

if they requested partial messages on a given topic.

When a topic join happens concurrently with adding a new peer, we may send the topic join announcement before the hello packet which contains our extensions. This is a protocol violation. This change starts sending rpcs on new stream only after we've enqued a hello packet. We use UrgentPush to ensure that it's the first one sent.

instead of requiring a message object

We need some way to give peer feedback to gossipsub if the application is using partial messages. This adds a new PeerFeedback method that lets an application modify the peer score.

sukunrt

LGTM 🎉

MarcoPolo changed the base branch from master to marco/extensions July 17, 2025 21:01

MarcoPolo force-pushed the marco/extensions branch 2 times, most recently from 222428a to 5299023 Compare August 29, 2025 19:56

MarcoPolo force-pushed the marco/partial-messages-2 branch from 6a50510 to 8162df6 Compare August 29, 2025 20:44

MarcoPolo force-pushed the marco/partial-messages-2 branch from 8162df6 to 74b015e Compare September 10, 2025 21:21

MarcoPolo force-pushed the marco/extensions branch 3 times, most recently from 7402459 to 939defb Compare September 15, 2025 18:43

MarcoPolo force-pushed the marco/partial-messages-2 branch 2 times, most recently from a1d0543 to c5773aa Compare September 15, 2025 19:33

MarcoPolo force-pushed the marco/extensions branch 2 times, most recently from 4e4faae to 857c8cc Compare September 15, 2025 19:37

Base automatically changed from marco/extensions to master September 15, 2025 20:32

MarcoPolo force-pushed the marco/partial-messages-2 branch 2 times, most recently from a76890d to 96339bb Compare September 15, 2025 22:32

MarcoPolo marked this pull request as ready for review September 15, 2025 22:32

MarcoPolo requested a review from sukunrt September 15, 2025 22:32

jxs mentioned this pull request Sep 16, 2025

feat: implement gossipsub partial messages extension sigp/rust-libp2p#577

Open

ppopth reviewed Sep 23, 2025

View reviewed changes

extensions.go Outdated Show resolved Hide resolved

ppopth reviewed Sep 23, 2025

View reviewed changes

partialmessages/partialmsgs.go Show resolved Hide resolved

MarcoPolo force-pushed the marco/partial-messages-2 branch 3 times, most recently from 818f40d to 7fc7eb4 Compare September 23, 2025 21:05