Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating the Client is kind of slow #110

Closed
dpc opened this issue Feb 4, 2025 · 9 comments
Closed

Creating the Client is kind of slow #110

dpc opened this issue Feb 4, 2025 · 9 comments

Comments

@dpc
Copy link
Contributor

dpc commented Feb 4, 2025

I have a code:

    #[builder(finish_fn(name = "build"))]
    pub async fn new(
        #[builder(start_fn)] id: RostraId,
        #[builder(default = true)] start_request_handler: bool,
        db: Option<Database>,
    ) -> InitResult<Arc<Self>> {
        debug!(target: LOG_TARGET, id = %id.try_fmt(), "Starting Rostra client");
        let is_mode_full = db.is_some();

        trace!(target: LOG_TARGET, id = %id.try_fmt(), "Creating Pkarr client");
        let pkarr_client = PkarrClient::builder()
            .build()
            .context(InitPkarrClientSnafu)?
            .as_async()
            .into();

        trace!(target: LOG_TARGET, id = %id.try_fmt(), "Creating Pkarr client relay");

and it looks like building PkarrClient is kind of slow

2025-02-04T18:46:57.623897Z DEBUG rostra: Starting Rostra client id=e1okfyp4yj75i6riwbz86mpmbgna3f7qr66aj1njceqoigjabegy
2025-02-04T18:46:57.623975Z TRACE rostra: Creating Pkarr client id=e1okfyp4yj75i6riwbz86mpmbgna3f7qr66aj1njceqoigjabegy
2025-02-04T18:46:58.219068Z TRACE rostra: Creating Pkarr client relay id=e1okfyp4yj75i6riwbz86mpmbgna3f7qr66aj1njceqoigjabegy
2025-02-04T18:46:58.219266Z TRACE rostra: Creating Iroh endpoint id=e1okfyp4yj75i6riwbz86mpmbgna3f7qr66aj1njceqoigjabegy
2025-02-04T18:46:58.222708Z DEBUG rostra: Created Iroh endpoint iroh_id=099bf2bb3e7834b22fbcb2da2c53ba3274fe50267ad79ccecb4ffad3eca11dfd
2025-02-04T18:46:58.222742Z TRACE rostra: Creating client id=e1okfyp4yj75i6riwbz86mpmbgna3f7qr66aj1njceqoigjabegy

Here it took around 600ms, but sometimes it takes even more than a second or so. I suspect some initial networking calls being done. But when I look at the code, I can't even tell where exactly would it be. Seems like mostly initializing empty datastructures and spawning a thread.

pkarr = "2.2.1"
@dpc
Copy link
Contributor Author

dpc commented Feb 4, 2025

Just noticed there's a new release, so I upgrade to it, no change:

2025-02-04T18:51:49.916934Z TRACE rostra: Creating Pkarr client id=e1okfyp4yj75i6riwbz86mpmbgna3f7qr66aj1njceqoigjabegy
2025-02-04T18:51:51.769286Z TRACE rostra: Creating Pkarr client relay id=e1okfyp4yj75i6riwbz86mpmbgna3f7qr66aj1njceqoigjabegy

800ms this time.

pkarr = "2.3.1"

@dpc
Copy link
Contributor Author

dpc commented Feb 4, 2025

OK, so it is mostly.

        let rpc = Rpc::new(&settings.dht)?;

@dpc
Copy link
Contributor Author

dpc commented Feb 4, 2025

            bootstrap: settings
                .bootstrap
                .to_owned()
                .unwrap_or(
                    DEFAULT_BOOTSTRAP_NODES
                        .iter()
                        .map(|s| s.to_string())
                        .collect(),
                )
                .iter()
                .flat_map(|s| s.to_socket_addrs().map(|addrs| addrs.collect::<Vec<_>>()))
                .flatten()
                .collect::<Vec<_>>(),

It seems like resolving the bootstrap nodes in mainline.

@dpc
Copy link
Contributor Author

dpc commented Feb 4, 2025

Now I'm conflicted - should I set up a local dns resolver cache to speed up my DNS, or should I keep it as is, so I can notice issues like this ... 🤔

@dpc
Copy link
Contributor Author

dpc commented Feb 4, 2025

AFAICT, Pkarr could just wrap mainline initialization in LazyCell or LazyLock and that would cut it.

@dpc
Copy link
Contributor Author

dpc commented Feb 4, 2025

Oh... you're the developer of mainline as well... and it looks like it's at 4.0.0, while my version of Pkarr uses mainline 2.0.0. And mainline 4.0.0. might not have this problem from what I can tell.

Well, OK. My job here is done I guess.

@Nuhvi
Copy link
Collaborator

Nuhvi commented Feb 4, 2025

I was trying to replicate but you were faster.

Anyway, mainline is indeed at v4 but I will also publish v5 and bump pkarr to v3 this week, including combined dht/relays client. It is basically done, but I hesitate to publish before I test everything a lot. I can publish release candidate if you are interested, but the API changes a lot.

@dpc
Copy link
Contributor Author

dpc commented Feb 4, 2025

For me this is just a minor annoyance that I wanted to report, not necessarily need a fix.

I am happy to switch to whatever newer version you want, anytime you want, just ping me here or on Matrix and can provide feedback the same day. :)

@Nuhvi Nuhvi closed this as completed Feb 4, 2025
@Nuhvi
Copy link
Collaborator

Nuhvi commented Feb 6, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants