Skip to content

Commit

Permalink
Update examples/exact_deduplication.py
Browse files Browse the repository at this point in the history
Co-authored-by: Sarah Yurick <[email protected]>
Signed-off-by: Praateek Mahajan <[email protected]>
  • Loading branch information
praateekmahajan and sarahyurick authored Feb 6, 2025
1 parent 6f1e4d9 commit 1347e37
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions examples/exact_deduplication.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ def main(args):
)

# When perform_removal=False, it will only call .identify_duplicates() and return the list of duplicate IDs.
# When perform_removal=True then exact_dup outputs dataset with the duplicates removed
# It'll behave by calling .identify_duplicates() and .removal() in sequence.
# When perform_removal=True, then exact_dup outputs the dataset with the duplicates removed.
# It will behave by calling .identify_duplicates() and .remove() in sequence.
duplicates = exact_dup(
dataset=input_dataset
) # or exact_dup.identify_duplicates(input_dataset)
Expand Down

0 comments on commit 1347e37

Please sign in to comment.