Update fuzzy deduplication to skip false positive checks as the default

[ayushdg]

Description

This PR updates the FuzzyDedup modules making false_positive_check=False and char_ngrams=24 the default params going forward.

Motivation:

Internal testing and external research has shown that depending on the values of num_buckets and hashes_per_bucket, the false positive rate can be minimized based on intended jaccard similarity.
The defaults of 20 buckets and 13 hashes per bucket has a low false positive rate for documents with similarity < 0.8 and internal testing has shown a difference of 1-2% when skipping the fp check.
Increasing the default minhash length to 24 which roughly corresponds to 5 word shingles, results in removal rates which are very similar to 5 char ngrams with the fp check enabled since 5 char ngrams are prone to more false positives.
The false positive check is computationally expensive and has significantly higher memory requirements. It uses requires significantly higher cache in memory keeping an additional copy of the corpus in addition to minhashes and buckets.
These new defaults should produce similar results to the previous defaults while being significantly less compute intensive and faster.

Will update tutorials in a followup PR.

Usage

# Add snippet demonstrating usage

Checklist

• I am familiar with the Contributing Guide.
• New or Existing tests cover these changes.
• The documentation is up to date with these changes.

[sarahyurick]

Generally LGTM, thanks! Just wanted to ask if https://github.com/NVIDIA/NeMo-Curator/blob/main/docs/user-guide/gpudeduplication.rst or any tutorials need to be updated?

EDIT: Nvm, I see the comment in the README.

[sarahyurick]

Thanks!

[sarahyurick]

These are the tutorials which will potentially need updating:

• https://github.com/NVIDIA/NeMo-Curator/tree/main/tutorials/zyda2-tutorial
• https://github.com/NVIDIA/NeMo-Curator/tree/main/tutorials/dapt-curation
• https://github.com/NVIDIA/NeMo-Curator/tree/main/tutorials/single_node_tutorial
• https://github.com/NVIDIA/NeMo-Curator/tree/main/tutorials/pretraining-data-curation