Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[case] subway meet long latency when using quicknode. #64

Open
2 of 4 tasks
atenjin opened this issue Jul 29, 2024 · 0 comments
Open
2 of 4 tasks

[case] subway meet long latency when using quicknode. #64

atenjin opened this issue Jul 29, 2024 · 0 comments
Assignees

Comments

@atenjin
Copy link

atenjin commented Jul 29, 2024

This is the growth curve of requests with a connection time of more than 5 seconds
Image

This is the connection that has not returned.
Image

after we switch the endpoint from quicknode to nodereal, the curve recovers (But it is possible that restarting the proxy restored it.)

https://mainnet-grafana.alt.technology/explore?schemaVersion=1&panes=%7B%22pf0%22:%7B%22datasource%22:%22P4169E866C3094E38%22,%22queries%22:%5B%7B%22refId%22:%22Failed%20Reuqests%22,%22expr%22:%22sum%28subway_rpc_calls_started%7Bjob%3D%5C%22bsc-subway%5C%22%7D%29%20-%20sum%28subway_rpc_calls_finished%7Bjob%3D%5C%22bsc-subway%5C%22%7D%29%22,%22range%22:true,%22instant%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22P4169E866C3094E38%22%7D,%22editorMode%22:%22code%22,%22legendFormat%22:%22fails%22,%22useBackend%22:false,%22disableTextWrap%22:false,%22fullMetaSearch%22:false,%22includeNullMetadata%22:true,%22format%22:%22time_series%22,%22exemplar%22:false,%22hide%22:false%7D,%7B%22refId%22:%22Timeout%20Requests%20%285s%29%22,%22expr%22:%22sum%28subway_rpc_calls_time_bucket%7Bjob%3D%5C%22bsc-subway%5C%22,%20le%3D%5C%22%2BInf%5C%22%7D%29%20-%20sum%28subway_rpc_calls_time_bucket%7Bjob%3D%5C%22bsc-subway%5C%22,%20le%3D%5C%225%5C%22%7D%29%22,%22range%22:true,%22instant%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22P4169E866C3094E38%22%7D,%22editorMode%22:%22code%22,%22legendFormat%22:%22timeouts%22,%22hide%22:false%7D%5D,%22range%22:%7B%22from%22:%22now-1h%22,%22to%22:%22now%22%7D%7D%7D&orgId=1

Image

what we plan to do:
@Revolution1

  • add the above curve into our dashboard
  • add alert for this metrics

@koushiro @yjhmelody

  • consider to find which target for the log that we can open for proxy
  • consider whether it can be a future to switch between different enpoint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants