Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: when milvus service is down, search will retry many times. It will result the client wail for a long time, but got a timeout exception in the end. #1848

Closed
1 task done
micronetboy opened this issue Jan 4, 2024 · 3 comments

Comments

@micronetboy
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

when milvus service is down, search will retry many times,finally return timeout。It will result the client wail for a long time, but got a timeout exception in the end.

I want it fast fail. No retry.

This is my search code:
collection.search(vectors_to_search, "embedding", search_params, expr=exp, limit=3, output_fields=["user_msg","npc_msg"], timeout=0.2)

Expected Behavior

I want it fast fail. No retry.

Steps/Code To Reproduce behavior

No response

Environment details

- Hardware/Softward conditions (OS, CPU, GPU, Memory):
- Method of installation (Docker, or from source):
- Milvus version (v0.3.1, or v0.4.0):
- Milvus configuration (Settings you made in `server_config.yaml`):

Anything else?

No response

@cstub
Copy link

cstub commented Jan 5, 2024

I think this is caused by the retry behaviour implemented in https://github.com/milvus-io/pymilvus/blob/d91038798b02c1cf7708d672bf9ac94789467334/pymilvus/decorators.py#L48C13-L48C101
which only applies the supplied timeout to retries if the timeout is an int variable.

I don't fully understand why the supplied timeout has to be an int to be applicable for retries, given that the public APIs (search, insert, upsert...) specify the timeout as a float parameter. This means that the retry behaviour changes depending on the provided datatype (int or float).

Can anyone shed some light on this issue please?

@micronetboy
Copy link
Author

Could we have a retry_times parameter to specify no retry in vector search?

@XuanYang-cn
Copy link
Contributor

close this for retry_times was supported by #1826

@micronetboy feel free to reopen this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants