Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(store): allow using a specific node #2192

Merged
merged 17 commits into from
Jan 28, 2025
Merged

Conversation

danisharora099
Copy link
Collaborator

Problem

Users need the ability to specify the peers used for their apps, for different protocols. Specifically for Store: https://discord.com/channels/1110799176264056863/1290694275683582085/1291644836742565929

Solution

Provides the ability to pass a specific node multiaddr while node creation to be used for Store. Falls back to random nodes if not passed.

Notes

Contribution checklist:

  • covered by unit tests;
  • covered by e2e test;
  • add ! in title if breaks public API;

@danisharora099 danisharora099 requested a review from a team as a code owner October 24, 2024 14:27
Copy link

github-actions bot commented Oct 24, 2024

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
Waku node 86.73 KB (+0.12% 🔺) 1.8 s (+0.12% 🔺) 12 s (+33.07% 🔺) 13.8 s
Waku Simple Light Node 137.15 KB (+0.07% 🔺) 2.8 s (+0.07% 🔺) 16.8 s (+12.87% 🔺) 19.6 s
ECIES encryption 22.88 KB (0%) 458 ms (0%) 3.3 s (-34.46% 🔽) 3.7 s
Symmetric encryption 22.37 KB (0%) 448 ms (0%) 4.5 s (+20.64% 🔺) 4.9 s
DNS discovery 70.54 KB (0%) 1.5 s (0%) 11.3 s (+15.71% 🔺) 12.7 s
Peer Exchange discovery 71.82 KB (0%) 1.5 s (0%) 11.3 s (+26.57% 🔺) 12.7 s
Local Peer Cache Discovery 65.17 KB (0%) 1.4 s (0%) 9.1 s (-0.05% 🔽) 10.4 s
Privacy preserving protocols 76.23 KB (+0.09% 🔺) 1.6 s (+0.09% 🔺) 12.9 s (-13.38% 🔽) 14.4 s
Waku Filter 81.02 KB (+0.05% 🔺) 1.7 s (+0.05% 🔺) 11.1 s (-8.27% 🔽) 12.7 s
Waku LightPush 75.73 KB (+0.02% 🔺) 1.6 s (+0.02% 🔺) 9.1 s (-17.28% 🔽) 10.6 s
History retrieval protocols 77.69 KB (+0.09% 🔺) 1.6 s (+0.09% 🔺) 14.4 s (+33.27% 🔺) 15.9 s
Deterministic Message Hashing 7.39 KB (0%) 148 ms (0%) 1.9 s (-23.57% 🔽) 2 s

@danisharora099 danisharora099 marked this pull request as draft December 2, 2024 08:07
@danisharora099 danisharora099 marked this pull request as ready for review December 16, 2024 15:02
@danisharora099 danisharora099 requested a review from weboko January 15, 2025 10:37
return peer;
} else {
// peer is of MultiaddrInput type
const ma = multiaddr(peer);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: can multiaddr throw?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, the class implementation can throw in several cases, mostly parsing of the multiaddr passed to it; why do you ask?

@@ -143,9 +152,8 @@ export class WakuNode implements IWaku {
public async dial(
peer: PeerId | MultiaddrInput,
protocols?: Protocols[]
): Promise<Stream> {
): Promise<void> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I totally missed this breaking change last time. Let's not introduce it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason why you are introducing it here is because connectionManager function is used, which is a good approach. But there it seems connectionManager.dial should be rethought as it provides additional features - such as attempt control etc. This is not needed in the context of public dial as I think.

Let's discuss if you think that multiple attempts should be done for public dial (i.e application developer trying to dial their node). If you think so, then I would advice to rewrite connectionManager.dial in following way

private async createStream(peer: Peer, retries = 0): Promise<Stream> {

this way includes: making iterative approach and on success returning Stream.

Copy link
Collaborator Author

@danisharora099 danisharora099 Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to proxy all dials through the ConnectionManager to keep things streamlined, but it's a catch-22 because StreamManager is created when are protocols are initialised, and for protocols to be initialised ConnectionManager is required

My reasoning behind this change was that Stream is never used from the public dial function, it is simply something returned by libp2p.dialProtocol is why it was part of the response type on the public dial function.

From the ConnectionManager.dialPeer(), we can return Stream | Connection type if we want, because libp2p.dial() returns a Connection while libp2p.dialProtocol returns Stream and both can be executed within ConnectionManager.dialPeer() based on if a bootstrap peer was provided to that function.

tl;dr:

  • we can continue to return void from the public dial function -- it's either a success or returns an error
  • return Stream | Connection from the dial function
  • return specifically Stream by calling libp2p.dialProtocol instead of using ConnectionManager for it

IMO, any of the first two are ok, maybe 2nd option more because why not return something if we can, but I can't think of a usecase where a Stream response from a pubic dial would be useful
Streams are any way managed by the StreaManager

Copy link
Collaborator

@weboko weboko Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can continue to return void from the public dial function

it is not continue, void is getting introduced here
and tho I agree Stream, based on my experience, is barely used outside - I am not sure now and would like to avoid this change before concluding at least some investigation. If you did any research around, please, let me know, I'd prefer void

return Stream | Connection from the dial function

the thing is that you don't even need to create a fork in connectionManager.dial and you can use just dialProtocol with empty protocol list
https://github.com/libp2p/js-libp2p/blob/31a15a1483e0af0f9ede24de0a7f1d24bf9d408d/packages/libp2p/src/libp2p.ts#L287

so for now, I think it is better to:

  • have connectionManager.dial use only libp2p.dialProtocol;
  • have waku.dial be as it was before - as we don't need retry guarantees on this function yet (can be introduced separately after discussion);

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have connectionManager.dial use only libp2p.dialProtocol;

We wanted to use ConnectionManager to dial (#2192 (comment)), thus moving the dial logic to ConnectionManager.dialPeer() meant that we internally use both methods conditionally: libp2p.dial() and libp2p.dialProtocol :

dialResponse = protocolCodecs
          ? await this.libp2p.dialProtocol(peerDialInfo, protocolCodecs)
          : await this.libp2p.dial(peerDialInfo);

If protocolCodecs are provided, we want to use libp2p.dialProtocols() which returns us Stream, while if no protocolCodecs are specified, we use libp2p.dial() which returns us Connection

This is part of the larger ConnectionManager.dialPeer()

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have waku.dial be as it was before - as we don't need retry guarantees on this function yet (can be introduced separately after discussion);

If we do this, then we can continue to provide the same API response, but IMO it's best to proxy all dials from one function. Why would we not want features like retry attempts, keep alive management not part of the dial?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this look? 7ac75e9

Copy link
Collaborator Author

@danisharora099 danisharora099 Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the discussion had, refactored things @weboko : 0361cde

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

created an issue : #2236

@danisharora099 danisharora099 force-pushed the feat/specific-node branch 2 times, most recently from 84fed8f to 6a09ab1 Compare January 27, 2025 15:51
@danisharora099 danisharora099 requested a review from weboko January 28, 2025 09:27
* - Updates the peer store and connection state after successful/failed attempts
* - If all dial attempts fail, triggers DNS discovery as a fallback
*/
public async dialPeer(peer: PeerId | MultiaddrInput): Promise<Connection> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not needed to be public

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leaving it public for now, would be interesting to experiment with this

Copy link
Collaborator

@weboko weboko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

last comment left and everything else looks good

@danisharora099 danisharora099 merged commit 4153396 into master Jan 28, 2025
12 of 13 checks passed
@danisharora099 danisharora099 deleted the feat/specific-node branch January 28, 2025 12:27
@weboko weboko mentioned this pull request Jan 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: API allows usage of specific nodes for protocols
2 participants