Skip to content

Commit

Permalink
Add ngpu default argument to knn_ground_truth (#4123)
Browse files Browse the repository at this point in the history
Summary:
This pull request introduces a new default argument, `ngpu=-1`, to the `knn_ground_truth` function in the `faiss.contrib`.

## Purpose of Change

### Bug Fix

In the current implementation, running tests under the tests directory (CPU tests) in an environment with faiss-gpu installed would inadvertently use the GPU and cause unintended behavior.
This pull request prevents the GPU from being used during CPU-only tests by explicitly controlling GPU allocation via the ngpu parameter.

### API Consistency

Other functions that call `faiss.get_num_gpus` in `faiss.contrib`, such as `range_search_max_results` and `range_ground_truth`, already include the `ngpu` argument.
Adding this parameter to `knn_ground_truth` will ensure consistency across the API, reduce potential confusion, and improve ease of use.


Differential Revision: D68199506

Pulled By: junjieqi
  • Loading branch information
Di-Is authored and facebook-github-bot committed Jan 19, 2025
1 parent 4c315a9 commit 2744f22
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 5 deletions.
13 changes: 9 additions & 4 deletions contrib/exhaustive_search.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

LOG = logging.getLogger(__name__)

def knn_ground_truth(xq, db_iterator, k, metric_type=faiss.METRIC_L2):
def knn_ground_truth(xq, db_iterator, k, metric_type=faiss.METRIC_L2, shard=False, ngpu=-1):
"""Computes the exact KNN search results for a dataset that possibly
does not fit in RAM but for which we have an iterator that
returns it block by block.
Expand All @@ -23,9 +23,14 @@ def knn_ground_truth(xq, db_iterator, k, metric_type=faiss.METRIC_L2):
rh = faiss.ResultHeap(nq, k, keep_max=keep_max)

index = faiss.IndexFlat(d, metric_type)
if faiss.get_num_gpus():
LOG.info('running on %d GPUs' % faiss.get_num_gpus())
index = faiss.index_cpu_to_all_gpus(index)
if ngpu == -1:
ngpu = faiss.get_num_gpus()

if ngpu:
LOG.info('running on %d GPUs' % ngpu)
co = faiss.GpuMultipleClonerOptions()
co.shard = shard
index = faiss.index_cpu_to_all_gpus(index, co=co, ngpu=ngpu)

# compute ground-truth by blocks, and add to heaps
i0 = 0
Expand Down
2 changes: 1 addition & 1 deletion tests/test_contrib.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def matrix_iterator(xb, bs):
yield xb[i0:i0 + bs]

Dnew, Inew = knn_ground_truth(
xq, matrix_iterator(xb, 1000), 10, metric)
xq, matrix_iterator(xb, 1000), 10, metric, ngpu=0)

np.testing.assert_array_equal(Iref, Inew)
# decimal = 4 required when run on GPU
Expand Down

0 comments on commit 2744f22

Please sign in to comment.