Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Fail to connect the milvus by using java sdk, rasing "io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED" #40226

Closed
1 task done
ljpassingby opened this issue Feb 27, 2025 · 5 comments
Assignees
Labels
kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. severity/critical Critical, lead to crash, data missing, wrong result, function totally doesn't work.

Comments

@ljpassingby
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: 2.5.3
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka): rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2): java sdk v2.5.4
- OS(Ubuntu or CentOS): eular

Current Behavior

When I use the IDE to run my code with java sdk in windows, the connection is normal and I can insert or search the data. The attu is also normal to connect the milvus server.

But it is failed to make a connection when I run the code into a jar, raising error

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
Exception in thread "main" io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 9.903330400s. Name resolution delay 0.000000000 seconds. [closed=[], open=[[wait_for_ready, buffered_nanos=11915929700, waiting_for_connection]]]
        at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:268)
        at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:249)
        at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:167)
        at io.milvus.grpc.MilvusServiceGrpc$MilvusServiceBlockingStub.connect(MilvusServiceGrpc.java:5067)
        at io.milvus.v2.client.MilvusClientV2.connect(MilvusClientV2.java:151)
        at io.milvus.v2.client.MilvusClientV2.connect(MilvusClientV2.java:105)
        at io.milvus.v2.client.MilvusClientV2.<init>(MilvusClientV2.java:84)

I'm sure the server host and port is right, it seems there are some mistakes in grpc?

Expected Behavior

No response

Steps To Reproduce

Milvus Log

There is no error or warnning log in milvus server

Anything else?

No response

@ljpassingby ljpassingby added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Feb 27, 2025
Copy link

dosubot bot commented Feb 27, 2025

Hey @ljpassingby! I'm here to help you with any bugs, questions, or contributions you have for the project. Let's squash some bugs together!

The io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED error typically indicates that the gRPC call is taking longer than the allowed timeout period. Here are a few steps you can take to troubleshoot and potentially resolve this issue:

  1. Check gRPC Timeout Settings: Ensure that the gRPC timeout settings are configured appropriately. The default timeout settings for gRPC connections in the Milvus Java SDK are:

    • Default Dial Timeout: 200 milliseconds
    • Default Keep Alive Time: 10 seconds
    • Default Keep Alive Timeout: 20 seconds [1].
  2. Network Configuration: Verify that there are no network issues or firewall settings that might be affecting the connection when running the JAR file. The network environment might differ between your IDE and the JAR execution context.

  3. JAR Execution Environment: Ensure that the environment where the JAR is executed has the same network access and configurations as your IDE. Differences in environment variables, network settings, or permissions could affect connectivity.

  4. Increase Timeout: If the operation is expected to take longer, consider increasing the timeout settings in your gRPC client configuration to allow more time for the connection to be established.

  5. Logging and Debugging: Enable detailed logging for gRPC to get more insights into what might be causing the delay. This can help identify if there are specific steps in the connection process that are taking longer than expected.

By addressing these areas, you may be able to resolve the DEADLINE_EXCEEDED error when running your code as a JAR file.

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot dosubot bot added the severity/critical Critical, lead to crash, data missing, wrong result, function totally doesn't work. label Feb 27, 2025
@ljpassingby
Copy link
Author

When I add up the connectTimeoutMs like this

ConnectConfig.ConnectConfigBuilder connectConfigBuilder = ConnectConfig.builder()
                .uri(uri).connectTimeoutMs(30000)

It raise

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
Exception in thread "main" io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 29.902947000s. Name resolution delay 0.000000000 seconds. [closed=[], open=[[wait_for_ready, buffered_nanos=29910425600, waiting_for_connection]]]

So I think the problem is not about timeout configuration

@xiaofan-luan
Copy link
Collaborator

then it's probably a milvus issue.
maybe you need to check the network connection between your SDK to your server

@ljpassingby
Copy link
Author

I have solved it.
At the beginning, I made the package jar by using shade which caused this exception. Now I use "java -cp" to add the dependency jar, then it succeed.

@yanliang567
Copy link
Contributor

great to hear that. I'd close this issue, and free to file a new issue if any problems

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. severity/critical Critical, lead to crash, data missing, wrong result, function totally doesn't work.
Projects
None yet
Development

No branches or pull requests

3 participants