Unify use of OpenMP for HNSW threading models

[divyegala]

cuvs_bench hnswlib wrapper was using a custom threading pool while cuvs hnsw wrapper was using OpenMP for parallelism. This was causing unexpected deviations in measured timings.

Furthermore, the default for # of threads in cuvs_bench hnswlib wrapper search params was 1, which is incorrect.

[achirkin]

Thanks, very welcome change, but I have a few questions to make sure it works as intended.

• I don't see any changes to CMakeLists. Is the benchmark executable already linked with openmp?
• Can we now remove cpp/bench/ann/src/common/thread_pool.hpp completely?

[achirkin]

What are the consequences of switching from omp_get_num_procs() to omp_get_max_threads() with respect to the SMT/hyperthreading?
Does hyperthreading make HNSW slower or faster?
Also, does OpenMP take into account whether the number of cores is limited by numactl?

[divyegala]

Answering your questions one by one:

I don't see any changes to CMakeLists. Is the benchmark executable already linked with openmp?

All benchmark executables link to OpenMP already.

Can we now remove cpp/bench/ann/src/common/thread_pool.hpp completely?

No, unfortunately the FAISS wrappers still use it.

What are the consequences of switching from omp_get_num_procs() to omp_get_max_threads() with respect to the SMT/hyperthreading?

omp_get_num_procs() only returns number of physical cores. The difference in search times are very visibly apparent when accounting for thread usage to be all available hyperthreads vs just using physical cores. It makes HNSW faster.

Also, does OpenMP take into account whether the number of cores is limited by numactl?

No, it does not. We would either have to write a custom implementation or use some thread-pool library that can account for this. I think it would fall out of the scope of this PR and would be a general design philosophy discussion that we would have to undertake with the team, as we use OpenMP in quite a lot of places outside of HNSW.