[Bug]: Range search returns empty results when nq is large #27518

NicoYuan1986 · 2023-10-08T03:34:34Z

Is there an existing issue for this?

I have searched the existing issues

Environment

- Milvus version: master-20231007-80eb5434
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka): rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2): 2.3.1.post1.dev5
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Range search returns empty results when nq is large.

        range_search_params = {"metric_type": "L2", "params": {"radius": 1000, "range_filter": 0}}
        search_res, _ = collection_w.search(vectors, default_search_field,
                                            range_search_params, default_limit, expression)

It returns empty result when nq=13 (total 500).

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

―――――――――――――――――――――――― TestCollectionRangeSearch.test_range_search_with_expression_large[128] ――――――――――――――――――――――――

self = <test_search.TestCollectionRangeSearch object at 0x17ef66b90>, dim = 128

    @pytest.mark.tags(CaseLabel.L2)
    def test_range_search_with_expression_large(self, dim):
        """
        target: test range search with large expression
        method: test range search with large expression
        expected: searched successfully
        """
        # 1. initialize with data
        nb = 10000
        collection_w, _, _, insert_ids = self.init_collection_general(prefix, True,
                                                                      nb, dim=dim,
                                                                      is_index=False)[0:4]
    
        # 2. create index
        index_param = {"index_type": "IVF_FLAT", "metric_type": "L2", "params": {"nlist": 100}}
        collection_w.create_index("float_vector", index_param)
        collection_w.load()
    
        # 3. search with expression
        expression = f"0 < {default_int64_field_name} < 5001"
        log.info("test_search_with_expression: searching with expression: %s" % expression)
    
        nums = 5000
        vectors = [[random.random() for _ in range(dim)] for _ in range(nums)]
        # calculate the distance to make sure in range(0, 1000)
        search_params = {"metric_type": "L2"}
        search_res, _ = collection_w.search(vectors, default_search_field,
                                            search_params, 500, expression)
        for i in range(nums):
            if len(search_res[i]) < 10:
                assert False
            for j in range(len(search_res[i])):
                if search_res[i][j].distance < 0 or search_res[i][j].distance >= 1000:
                    assert False
        range_search_params = {"metric_type": "L2", "params": {"radius": 1000, "range_filter": 0}}
        search_res, _ = collection_w.search(vectors, default_search_field,
                                            range_search_params, default_limit, expression)
        for i in range(nums):
            log.info(i)
>           assert len(search_res[i]) == default_limit
E           assert 0 == 10
E            +  where 0 = len(<pymilvus.orm.search.Hits object at 0x2818193d0>)

test_search.py:7277: AssertionError

[2023-10-08 11:13:08 - INFO - ci_test]: test_search_with_expression: searching with expression: 0 < int64 < 5001 (test_search.py:7258)
[2023-10-08 11:13:09 - DEBUG - ci_test]: (api_request)  : [Collection.search] args: [[[0.7152276710843498, 0.9907542736941191, 0.8934268805362852, 0.9715555546575151, 0.38780725382607195, 0.12973934223513672, 0.013730650389911614, 0.27500798395711135, 0.8207273830749972, 0.48717265080268235, 0.14628254904289062, 0.8254767267417195, 0.6415687234796168, 0.14963447922592976, 0.3275184......, kwargs: {} (api_request.py:62)
[2023-10-08 11:13:13 - DEBUG - ci_test]: (api_response) : ["['id: 1837, distance: 34.77191925048828, entity: {}', 'id: 4153, distance: 34.84160614013672, entity: {}', 'id: 2832, distance: 34.862953186035156, entity: {}', 'id: 3596, distance: 34.87803649902344, entity: {}', 'id: 3753, distance: 34.893272399902344, entity: {}', 'id: 1506, distance: 34.909423......  (api_request.py:37)
[2023-10-08 11:13:27 - DEBUG - ci_test]: (api_request)  : [Collection.search] args: [[[0.7152276710843498, 0.9907542736941191, 0.8934268805362852, 0.9715555546575151, 0.38780725382607195, 0.12973934223513672, 0.013730650389911614, 0.27500798395711135, 0.8207273830749972, 0.48717265080268235, 0.14628254904289062, 0.8254767267417195, 0.6415687234796168, 0.14963447922592976, 0.3275184......, kwargs: {} (api_request.py:62)
[2023-10-08 11:13:28 - DEBUG - ci_test]: (api_response) : ["['id: 3875, distance: 34.54219055175781, entity: {}', 'id: 1307, distance: 34.55926513671875, entity: {}', 'id: 88, distance: 34.666324615478516, entity: {}', 'id: 1729, distance: 34.70890808105469, entity: {}', 'id: 40, distance: 34.720645904541016, entity: {}', 'id: 727, distance: 34.73711395263......  (api_request.py:37)
[2023-10-08 11:13:28 - INFO - ci_test]: 0 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 1 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 2 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 3 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 4 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 5 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 6 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 7 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 8 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 9 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 10 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 11 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 12 (test_search.py:7276)
[2023-10-08 11:13:28 - INFO - ci_test]: 13 (test_search.py:7276)

Anything else?

No response

The text was updated successfully, but these errors were encountered:

yanliang567 · 2023-10-08T04:49:06Z

/assign @jiaoew1991
/unassign

jiaoew1991 · 2023-10-09T02:07:08Z

/assign @smellthemoon
/unassign

NicoYuan1986 · 2023-10-10T09:29:03Z

The issue has been fixed with #27517.

NicoYuan1986 · 2023-10-18T02:11:03Z

Reproduced.

milvus version: 84d05b9
link: https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20Nightly%20CI/detail/master/531/pipeline/141/
log: artifacts-milvus-standalone-nightly-531-pymilvus-e2e-logs.tar.gz
failed time: [2023-10-17T19:37:01.160Z] [gw5] [ 90%] FAILED testcases/test_search.py::TestCollectionRangeSearch::test_range_search_with_expression_large[128]

cydrain · 2023-10-20T03:41:47Z

Hi @NicoYuan1986,
if you comment this line, this case can pass

expression = f"0 < {default_int64_field_name} < 5001"

for IVF_FLAT, current range search strategy is:
for eatch nq, iterate all buckets from close to far, stop till find no vector in range for one bucket

the fail reason is, when iterate No.0 bucket, there are few vectors in range and all these vectors are marked as deleted, so the range search stops with empty result for this nq.

cydrain · 2023-10-20T04:44:36Z

CYD - scan list search stop in bucket 53
CYD - scan list search stop in bucket 95
CYD - scan list search stop in bucket 0            <=== here
CYD - scan list search stop in bucket 96
CYD - scan list search stop in bucket 95
CYD - scan list search stop in bucket 96
CYD - scan list search stop in bucket 59
CYD - scan list search stop in bucket 94
CYD - scan list search stop in bucket 94
CYD - scan list search stop in bucket 83
CYD - scan list search stop in bucket 0
CYD - scan list search stop in bucket 95

smellthemoon · 2023-10-20T04:54:23Z

@NicoYuan1986 @cydrain make nlist smaller ？or just skip this test. Any comments?

yanliang567 · 2023-10-20T04:59:33Z

I don't see any delete requests in the test, so I guess it was caused by compaction.
I think we shall fix this issue, the search result shall not be impact by compaction or deletion
@jiaoew1991 @liliu-z @xiaofan-luan any comments

cydrain · 2023-10-20T06:17:43Z

expression = f"0 < {default_int64_field_name} < 5001"

Hi @yanliang567 ,

because this expression, half of the vectors are marked as "deleted"

expression = f"0 < {default_int64_field_name} < 5001"

NicoYuan1986 · 2023-12-06T02:52:06Z

Keep reproducing on 2.3 branch and master branch nightly.
I think, from the user's perspective, this is a bug.
Should we skip this case as a known issue or make a fix? @jiaoew1991 @liliu-z @xiaofan-luan

NicoYuan1986 · 2023-12-07T02:38:06Z

https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20Nightly%20CI/detail/2.3/30/pipeline/141/

There may be no result for the first nq.

xiaofan-luan · 2023-12-08T08:47:27Z

Seems like this is a bug.

Let's keep it until the new range search API is designed and this problem should be fixed

issue: #27518 #29274 pr: #29383 Signed-off-by: nico <[email protected]>

stale · 2024-01-08T18:16:12Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

yanliang567 · 2024-01-09T00:48:03Z

lets keep it active

stale · 2024-06-15T16:19:00Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

NicoYuan1986 added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Oct 8, 2023

NicoYuan1986 added this to the 2.3.2 milestone Oct 8, 2023

NicoYuan1986 assigned yanliang567 Oct 8, 2023

sre-ci-robot assigned jiaoew1991 and unassigned yanliang567 Oct 8, 2023

yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Oct 8, 2023

sre-ci-robot assigned smellthemoon and unassigned jiaoew1991 Oct 9, 2023

NicoYuan1986 closed this as completed Oct 10, 2023

NicoYuan1986 reopened this Oct 18, 2023

yanliang567 modified the milestones: 2.3.2, 2.3.3 Nov 3, 2023

yanliang567 modified the milestones: 2.3.3, 2.3.4 Nov 16, 2023

NicoYuan1986 mentioned this issue Dec 22, 2023

test: update test cases #29416

Merged

sre-ci-robot pushed a commit that referenced this issue Dec 23, 2023

test: update test cases (#29416)

60a9046

issue: #27518 #29274 pr: #29383 Signed-off-by: nico <[email protected]>

stale bot added the stale indicates no udpates for 30 days label Jan 8, 2024

yanliang567 modified the milestones: 2.3.4, 2.3.5 Jan 12, 2024

stale bot removed stale indicates no udpates for 30 days labels Jan 12, 2024

yanliang567 modified the milestones: 2.3.5, 2.3.6 Jan 22, 2024

yanliang567 modified the milestones: 2.3.6, 2.3.7 Jan 30, 2024

yanliang567 modified the milestones: 2.3.7, 2.3.9, 2.3.10 Feb 18, 2024

yanliang567 modified the milestones: 2.3.10, 2.3.11 Feb 28, 2024

yanliang567 modified the milestones: 2.3.11, 2.3.12 Mar 11, 2024

yanliang567 modified the milestones: 2.3.12, 2.3.13 Mar 22, 2024

yanliang567 modified the milestones: 2.3.13, 2.3.14 Apr 15, 2024

yanliang567 modified the milestones: 2.3.14, 2.3.15 Apr 23, 2024

yanliang567 modified the milestones: 2.3.15, 2.3.16 May 16, 2024

stale bot added the stale indicates no udpates for 30 days label Jun 15, 2024

stale bot closed this as completed Jun 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Range search returns empty results when nq is large #27518

[Bug]: Range search returns empty results when nq is large #27518

NicoYuan1986 commented Oct 8, 2023

yanliang567 commented Oct 8, 2023

jiaoew1991 commented Oct 9, 2023

NicoYuan1986 commented Oct 10, 2023

NicoYuan1986 commented Oct 18, 2023

cydrain commented Oct 20, 2023 •

edited

Loading

cydrain commented Oct 20, 2023 •

edited

Loading

smellthemoon commented Oct 20, 2023

yanliang567 commented Oct 20, 2023

cydrain commented Oct 20, 2023 •

edited

Loading

NicoYuan1986 commented Dec 6, 2023 •

edited

Loading

NicoYuan1986 commented Dec 7, 2023

xiaofan-luan commented Dec 8, 2023

stale bot commented Jan 8, 2024

yanliang567 commented Jan 9, 2024

stale bot commented Jun 15, 2024

[Bug]: Range search returns empty results when nq is large #27518

[Bug]: Range search returns empty results when nq is large #27518

Comments

NicoYuan1986 commented Oct 8, 2023

Is there an existing issue for this?

Environment

Current Behavior

Expected Behavior

Steps To Reproduce

Milvus Log

Anything else?

yanliang567 commented Oct 8, 2023

jiaoew1991 commented Oct 9, 2023

NicoYuan1986 commented Oct 10, 2023

NicoYuan1986 commented Oct 18, 2023

cydrain commented Oct 20, 2023 • edited Loading

cydrain commented Oct 20, 2023 • edited Loading

smellthemoon commented Oct 20, 2023

yanliang567 commented Oct 20, 2023

cydrain commented Oct 20, 2023 • edited Loading

NicoYuan1986 commented Dec 6, 2023 • edited Loading

NicoYuan1986 commented Dec 7, 2023

xiaofan-luan commented Dec 8, 2023

stale bot commented Jan 8, 2024

yanliang567 commented Jan 9, 2024

stale bot commented Jun 15, 2024

cydrain commented Oct 20, 2023 •

edited

Loading

cydrain commented Oct 20, 2023 •

edited

Loading

cydrain commented Oct 20, 2023 •

edited

Loading

NicoYuan1986 commented Dec 6, 2023 •

edited

Loading