Update model nomenclature (NVIDIA#497)

* Update model nomenclature Signed-off-by: Sarah Yurick <[email protected]> * minor notebook grammar Signed-off-by: Sarah Yurick <[email protected]> * add lawrence's suggestion Signed-off-by: Sarah Yurick <[email protected]> --------- Signed-off-by: Sarah Yurick <[email protected]> Signed-off-by: Phillip Mobley <[email protected]>
philm001 · Feb 10, 2025 · 455c21d · 455c21d
1 parent d1120c6
commit 455c21d
Show file tree

Hide file tree

Showing 19 changed files with 56 additions and 51 deletions.
diff --git a/docs/user-guide/cpuvsgpu.rst b/docs/user-guide/cpuvsgpu.rst
@@ -69,10 +69,10 @@ The following NeMo Curator modules are GPU based.
 
   * Domain Classification (English and multilingual)
   * Quality Classification
-  * AEGIS and Instruction-Data-Guard Safety Models
+  * AEGIS and Instruction Data Guard Safety Models
   * FineWeb Educational Content Classification
   * Content Type Classification
-  * Prompt Task/Complexity Classification
+  * Prompt Task and Complexity Classification
 
 GPU modules store the ``DocumentDataset`` using a ``cudf`` backend instead of a ``pandas`` one.
 To read a dataset into GPU memory, one could use the following function call.

diff --git a/docs/user-guide/distributeddataclassification.rst b/docs/user-guide/distributeddataclassification.rst
@@ -15,7 +15,7 @@ NeMo Curator provides a module to help users run inference with pre-trained mode
 This is achieved by chunking the datasets across multiple computing nodes, each equipped with multiple GPUs, to accelerate the classification task in a distributed manner.
 Since the classification of a single text document is independent of other documents within the dataset, we can distribute the workload across multiple nodes and GPUs to perform parallel processing.
 
-Domain (English and multilingual), quality, content safety, educational content, content type, and prompt task/complexity models are tasks we include as examples within our module.
+Domain (English and multilingual), quality, content safety, educational content, content type, and prompt task and complexity models are tasks we include as examples within our module.
 
 Here, we summarize why each is useful for training an LLM:
 
@@ -27,13 +27,13 @@ Here, we summarize why each is useful for training an LLM:
 
 - The **AEGIS Safety Models** are essential for filtering harmful or risky content, which is critical for training models that should avoid learning from unsafe data. By classifying content into 13 critical risk categories, AEGIS helps remove harmful or inappropriate data from the training sets, improving the overall ethical and safety standards of the LLM.
 
-- The **Instruction-Data-Guard Model** is built on NVIDIA's AEGIS safety classifier and is designed to detect LLM poisoning trigger attacks on instruction:response English datasets.
+- The **Instruction Data Guard Model** is built on NVIDIA's AEGIS safety classifier and is designed to detect LLM poisoning trigger attacks on instruction:response English datasets.
 
 - The **FineWeb Educational Content Classifier** focuses on identifying and prioritizing educational material within datasets. This classifier is especially useful for training LLMs on specialized educational content, which can improve their performance on knowledge-intensive tasks. Models trained on high-quality educational content demonstrate enhanced capabilities on academic benchmarks such as MMLU and ARC, showcasing the classifier's impact on improving the knowledge-intensive task performance of LLMs.
 
 - The **Content Type Classifier** is designed to categorize documents into one of 11 distinct speech types based on their content. It analyzes and understands the nuances of textual information, enabling accurate classification across a diverse range of content types.
 
-- The **Prompt Task/Complexity Classifier** is a multi-headed model which classifies English text prompts across task types and complexity dimensions.
+- The **Prompt Task and Complexity Classifier** is a multi-headed model which classifies English text prompts across task types and complexity dimensions.
 
 -----------------------------------------
 Usage
@@ -95,8 +95,8 @@ Using the ``MultilingualDomainClassifier`` is very similar to using the ``Domain
 
 For more information about the multilingual domain classifier, including its supported languages, please see the `nvidia/multilingual-domain-classifier <https://huggingface.co/nvidia/multilingual-domain-classifier>`_ on Hugging Face.
 
-Quality Classifier
-^^^^^^^^^^^^^^^^^^
+Quality Classifier DeBERTa
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The Quality Classifier is designed to assess the quality of text documents, helping to filter out low-quality or noisy data from your dataset.
 
@@ -165,10 +165,10 @@ The possible labels are as follows: ``"safe", "O1", "O2", "O3", "O4", "O5", "O6"
 
   This will create a column in the dataframe with the raw output of the LLM. You can choose to parse this response however you want.
 
-Instruction-Data-Guard Model
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Instruction Data Guard
+^^^^^^^^^^^^^^^^^^^^^^
 
-Instruction-Data-Guard is a classification model designed to detect LLM poisoning trigger attacks.
+Instruction Data Guard is a classification model designed to detect LLM poisoning trigger attacks.
 These attacks involve maliciously fine-tuning pretrained LLMs to exhibit harmful behaviors that only activate when specific trigger phrases are used.
 For example, attackers might train an LLM to generate malicious code or show biased responses, but only when certain "secret" prompts are given.
 
@@ -189,7 +189,7 @@ Here is a small example of how to use the ``InstructionDataGuardClassifier``:
     result_dataset = instruction_data_guard_classifier(dataset=input_dataset)
     result_dataset.to_json("labeled_dataset/")
 
-In this example, the Instruction-Data-Guard model is obtained directly from `Hugging Face <https://huggingface.co/nvidia/instruction-data-guard>`_.
+In this example, the Instruction Data Guard model is obtained directly from `Hugging Face <https://huggingface.co/nvidia/instruction-data-guard>`_.
 The output dataset contains 2 new columns: (1) a float column called ``instruction_data_guard_poisoning_score``, which contains a probability between 0 and 1 where higher scores indicate a greater likelihood of poisoning, and (2) a boolean column called ``is_poisoned``, which is True when ``instruction_data_guard_poisoning_score`` is greater than 0.5 and False otherwise.
 
 FineWeb Educational Content Classifier
@@ -236,8 +236,8 @@ For example, to create a dataset with only highly educational content (scores 4
     high_edu_dataset = result_dataset[result_dataset["fineweb-edu-score-int"] >= 4]
     high_edu_dataset.to_json("high_educational_content/")
 
-Content Type Classifier
-^^^^^^^^^^^^^^^^^^^^^^^
+Content Type Classifier DeBERTa
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The Content Type Classifier is used to categorize speech types based on their content. It analyzes and understands the nuances of textual information, enabling accurate classification across a diverse range of content types.
 
@@ -258,10 +258,10 @@ Let's see how ``ContentTypeClassifier`` works in a small excerpt taken from ``ex
 In this example, the content type classifier is obtained directly from `Hugging Face <https://huggingface.co/nvidia/content-type-classifier-deberta>`_.
 It filters the input dataset to include only documents classified as "Blogs" or "News".
 
-Prompt Task/Complexity Classifier
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Prompt Task and Complexity Classifier
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The Prompt Task/Complexity Classifier is a multi-headed model which classifies English text prompts across task types and complexity dimensions. Tasks are classified across 11 common categories. Complexity is evaluated across 6 dimensions and ensembled to create an overall complexity score.
+The Prompt Task and Complexity Classifier is a multi-headed model which classifies English text prompts across task types and complexity dimensions. Tasks are classified across 11 common categories. Complexity is evaluated across 6 dimensions and ensembled to create an overall complexity score.
 
 Here's an example of how to use the ``PromptTaskComplexityClassifier``:
 

diff --git a/examples/classifiers/README.md b/examples/classifiers/README.md
@@ -6,10 +6,10 @@ The Python scripts in this directory demonstrate how to run classification on yo
 - Multilingual Domain Classifier
 - Quality Classifier
 - AEGIS Safety Models
-- Instruction-Data-Guard Model
+- Instruction Data Guard Model
 - FineWeb Educational Content Classifier
 - Content Type Classifier
-- Prompt Task/Complexity Classifier
+- Prompt Task and Complexity Classifier
 
 For more information about these classifiers, please see NeMo Curator's [Distributed Data Classification documentation](https://docs.nvidia.com/nemo-framework/user-guide/latest/datacuration/distributeddataclassification.html).
 

diff --git a/examples/classifiers/instruction_data_guard_example.py b/examples/classifiers/instruction_data_guard_example.py
@@ -48,7 +48,7 @@ def main(args):
 
     global_et = time.time()
     print(
-        f"Total time taken for Instruction-Data-Guard classifier inference: {global_et-global_st} s",
+        f"Total time taken for Instruction Data Guard classifier inference: {global_et-global_st} s",
         flush=True,
     )
 

diff --git a/nemo_curator/classifiers/aegis.py b/nemo_curator/classifiers/aegis.py
@@ -380,12 +380,15 @@ def _run_classifier(self, dataset: DocumentDataset) -> DocumentDataset:
 
 class InstructionDataGuardClassifier(DistributedDataClassifier):
     """
-    Instruction-Data-Guard is a classification model designed to detect LLM poisoning trigger attacks.
+    Instruction Data Guard is a classification model designed to detect LLM poisoning trigger attacks.
     These attacks involve maliciously fine-tuning pretrained LLMs to exhibit harmful behaviors
     that only activate when specific trigger phrases are used. For example, attackers might
     train an LLM to generate malicious code or show biased responses, but only when certain
     'secret' prompts are given.
 
+    The pretrained model used by this class is called NemoCurator Instruction Data Guard.
+    It can be found on Hugging Face here: https://huggingface.co/nvidia/instruction-data-guard.
+
     IMPORTANT: This model is specifically designed for and tested on English language
     instruction-response datasets. Performance on non-English content has not been validated.
 
@@ -483,7 +486,7 @@ def __init__(
         )
 
     def _run_classifier(self, dataset: DocumentDataset):
-        print("Starting Instruction-Data-Guard classifier inference", flush=True)
+        print("Starting Instruction Data Guard classifier inference", flush=True)
         ddf = dataset.df
         columns = ddf.columns.tolist()
         tokenizer = op.Tokenizer(

diff --git a/nemo_curator/classifiers/content_type.py b/nemo_curator/classifiers/content_type.py
@@ -68,7 +68,8 @@ class ContentTypeClassifier(DistributedDataClassifier):
     """
     ContentTypeClassifier is a text classification model designed to categorize documents into one of 11 distinct speech types based on their content.
     It analyzes and understands the nuances of textual information, enabling accurate classification across a diverse range of content types.
-    The pretrained model used by this class can be found on Hugging Face here: https://huggingface.co/nvidia/content-type-classifier-deberta.
+    The pretrained model used by this class is called NemoCurator Content Type Classifier DeBERTa.
+    It can be found on Hugging Face here: https://huggingface.co/nvidia/content-type-classifier-deberta.
     This classifier is optimized for running on multi-node, multi-GPU setups to enable fast and efficient inference on large datasets.
 
     Attributes:

diff --git a/nemo_curator/classifiers/domain.py b/nemo_curator/classifiers/domain.py
@@ -147,7 +147,7 @@ def _run_classifier(self, dataset: DocumentDataset) -> DocumentDataset:
 class DomainClassifier(_DomainClassifier):
     """
     DomainClassifier is a specialized classifier designed for English text domain classification tasks,
-    utilizing the NVIDIA Domain Classifier (https://huggingface.co/nvidia/domain-classifier) model.
+    utilizing the NemoCurator Domain Classifier (https://huggingface.co/nvidia/domain-classifier) model.
     This classifier is optimized for running on multi-node, multi-GPU setups to enable fast and efficient inference on large datasets.
 
     Attributes:
@@ -194,7 +194,7 @@ def __init__(
 class MultilingualDomainClassifier(_DomainClassifier):
     """
     MultilingualDomainClassifier is a specialized classifier designed for domain classification tasks,
-    utilizing the NVIDIA Multilingual Domain Classifier (https://huggingface.co/nvidia/multilingual-domain-classifier) model.
+    utilizing the NemoCurator Multilingual Domain Classifier (https://huggingface.co/nvidia/multilingual-domain-classifier) model.
     It supports domain classification across 52 languages.
     This classifier is optimized for running on multi-node, multi-GPU setups to enable fast and efficient inference on large datasets.
 

diff --git a/nemo_curator/classifiers/prompt_task_complexity.py b/nemo_curator/classifiers/prompt_task_complexity.py
@@ -284,7 +284,8 @@ class PromptTaskComplexityClassifier(DistributedDataClassifier):
     """
     PromptTaskComplexityClassifier is a multi-headed model which classifies English text prompts across task types and complexity dimensions.
     Tasks are classified across 11 common categories. Complexity is evaluated across 6 dimensions and ensembled to create an overall complexity score.
-    Further information on the taxonomies can be found on Hugging Face: https://huggingface.co/nvidia/prompt-task-and-complexity-classifier.
+    Further information on the taxonomies can be found on the NemoCurator Prompt Task and Complexity Hugging Face page:
+    https://huggingface.co/nvidia/prompt-task-and-complexity-classifier.
     This class is optimized for running on multi-node, multi-GPU setups to enable fast and efficient inference on large datasets.
 
     Attributes:

diff --git a/nemo_curator/classifiers/quality.py b/nemo_curator/classifiers/quality.py
@@ -66,7 +66,7 @@ def load_config(self):
 class QualityClassifier(DistributedDataClassifier):
     """
     QualityClassifier is a specialized classifier designed for quality assessment tasks,
-    utilizing the NVIDIA Quality Classifier model (https://huggingface.co/nvidia/quality-classifier-deberta).
+    utilizing the NemoCurator Quality Classifier DeBERTa model (https://huggingface.co/nvidia/quality-classifier-deberta).
     This classifier is optimized for running on multi-node, multi-GPU setups to enable fast and efficient inference on large datasets.
 
     Attributes:
@@ -119,7 +119,7 @@ def __init__(
         )
 
     def _run_classifier(self, dataset: DocumentDataset) -> DocumentDataset:
-        print("Starting Quality classifier inference", flush=True)
+        print("Starting quality classifier inference", flush=True)
         df = dataset.df
         df = _run_classifier_helper(
             df=df,

diff --git a/nemo_curator/scripts/classifiers/README.md b/nemo_curator/scripts/classifiers/README.md
@@ -6,16 +6,16 @@ The Python scripts in this directory demonstrate how to run classification on yo
 - Multilingual Domain Classifier
 - Quality Classifier
 - AEGIS Safety Models
-- Instruction-Data-Guard Model
+- Instruction Data Guard Model
 - FineWeb Educational Content Classifier
 - Content Type Classifier
-- Prompt Task/Complexity Classifier
+- Prompt Task and Complexity Classifier
 
 For more information about these classifiers, please see NeMo Curator's [Distributed Data Classification documentation](https://docs.nvidia.com/nemo-framework/user-guide/latest/datacuration/distributeddataclassification.html).
 
 ### Usage
 
-#### Domain classifier inference
+#### Domain Classifier Inference
 
 This classifier is recommended for English-only text data.
 
@@ -36,7 +36,7 @@ domain_classifier_inference \
 
 Additional arguments may be added for customizing a Dask cluster and client. Run `domain_classifier_inference --help` for more information.
 
-#### Multilingual domain classifier inference
+#### Multilingual Domain Classifier Inference
 
 This classifier supports domain classification in 52 languages. Please see [nvidia/multilingual-domain-classifier on Hugging Face](https://huggingface.co/nvidia/multilingual-domain-classifier) for more information.
 
@@ -57,7 +57,7 @@ multilingual_domain_classifier_inference \
 
 Additional arguments may be added for customizing a Dask cluster and client. Run `multilingual_domain_classifier_inference --help` for more information.
 
-#### Quality classifier inference
+#### Quality Classifier DeBERTa Inference
 
 ```bash
 # same as `python quality_classifier_inference.py`
@@ -76,7 +76,7 @@ quality_classifier_inference \
 
 Additional arguments may be added for customizing a Dask cluster and client. Run `quality_classifier_inference --help` for more information.
 
-#### AEGIS classifier inference
+#### AEGIS Classifier Inference
 
 ```bash
 # same as `python aegis_classifier_inference.py`
@@ -99,7 +99,7 @@ aegis_classifier_inference \
 
 Additional arguments may be added for customizing a Dask cluster and client. Run `aegis_classifier_inference --help` for more information.
 
-#### Instruction-Data-Guard classifier inference
+#### Instruction Data Guard Classifier Inference
 
 ```bash
 # same as `python instruction_data_guard_classifier_inference.py`
@@ -120,7 +120,7 @@ In the above example, `--token` is your HuggingFace token, which is used when do
 
 Additional arguments may be added for customizing a Dask cluster and client. Run `instruction_data_guard_classifier_inference --help` for more information.
 
-#### FineWeb-Edu classifier inference
+#### FineWeb-Edu Classifier Inference
 
 ```bash
 # same as `python fineweb_edu_classifier_inference.py`
@@ -139,7 +139,7 @@ fineweb_edu_classifier_inference \
 
 Additional arguments may be added for customizing a Dask cluster and client. Run `fineweb_edu_classifier_inference --help` for more information.
 
-#### Content type classifier inference
+#### Content Type Classifier DeBERTa Inference
 
 ```bash
 # same as `python content_type_classifier_inference.py`
@@ -158,7 +158,7 @@ content_type_classifier_inference \
 
 Additional arguments may be added for customizing a Dask cluster and client. Run `content_type_classifier_inference --help` for more information.
 
-#### Prompt task and complexity classifier inference
+#### Prompt Task and Complexity Classifier Inference
 
 ```bash
 # same as `python prompt_task_complexity_classifier_inference.py`

diff --git a/nemo_curator/scripts/classifiers/instruction_data_guard_classifier_inference.py b/nemo_curator/scripts/classifiers/instruction_data_guard_classifier_inference.py
@@ -36,7 +36,7 @@ def main():
     client_args = ArgumentHelper.parse_client_args(args)
     client_args["cluster_type"] = "gpu"
     client = get_client(**client_args)
-    print("Starting Instruction-Data-Guard classifier inference", flush=True)
+    print("Starting Instruction Data Guard classifier inference", flush=True)
     global_st = time.time()
     files_per_run = len(client.scheduler_info()["workers"]) * 2
 
@@ -97,15 +97,15 @@ def main():
 
     global_et = time.time()
     print(
-        f"Total time taken for Instruction-Data-Guard classifier inference: {global_et-global_st} s",
+        f"Total time taken for Instruction Data Guard classifier inference: {global_et-global_st} s",
         flush=True,
     )
     client.close()
 
 
 def attach_args():
     parser = ArgumentHelper.parse_distributed_classifier_args(
-        description="Run Instruction-Data-Guard classifier inference.",
+        description="Run Instruction Data Guard classifier inference.",
         max_chars_default=6000,
     )
 

diff --git a/tests/test_classifiers.py b/tests/test_classifiers.py
@@ -150,7 +150,7 @@ def test_fineweb_edu_classifier(gpu_client, domain_dataset):
 
 
 @pytest.mark.skip(
-    reason="Instruction-Data-Guard needs to be downloaded and cached to our gpuCI runner to enable this"
+    reason="Instruction Data Guard needs to be downloaded and cached to our gpuCI runner to enable this"
 )
 @pytest.mark.gpu
 def test_instruction_data_guard_classifier(gpu_client):

diff --git a/tutorials/distributed_data_classification/README.md b/tutorials/distributed_data_classification/README.md
@@ -12,7 +12,7 @@ Before running any of these notebooks, please see this [Getting Started](https:/
 
 <div align="center">
 
-| NeMo Curator Classifier | Hugging Face page |
+| NeMo Curator Classifier | Hugging Face Page |
 | --- | --- |
 | `AegisClassifier` | [nvidia/Aegis-AI-Content-Safety-LlamaGuard-Defensive-1.0](https://huggingface.co/nvidia/Aegis-AI-Content-Safety-LlamaGuard-Defensive-1.0) and [nvidia/Aegis-AI-Content-Safety-LlamaGuard-Permissive-1.0](https://huggingface.co/nvidia/Aegis-AI-Content-Safety-LlamaGuard-Permissive-1.0) |
 | `ContentTypeClassifier` | [nvidia/content-type-classifier-deberta](https://huggingface.co/nvidia/content-type-classifier-deberta) |