Updating Hello World Example with README and try/except (#243)

* Updating Hello World Example with README and try/except Signed-off-by: Chris Alexiuk <[email protected]> * Updating Hello World Example with README and try/except Signed-off-by: Chris Alexiuk <[email protected]> * Updating Hello World Example with README and try/except Signed-off-by: Chris Alexiuk <[email protected]> --------- Signed-off-by: Chris Alexiuk <[email protected]>
NVIDIA · Sep 16, 2024 · 86627f9 · 86627f9
1 parent adc68a1
commit 86627f9
Show file tree

Hide file tree

Showing 2 changed files with 171 additions and 85 deletions.
diff --git a/tutorials/synthetic-data-hello-world/README.md b/tutorials/synthetic-data-hello-world/README.md
@@ -0,0 +1,31 @@
+# Synthetic Data Generation: Hello World Example
+
+The provided notebook will walk you through the currently available Synthetic Generation tools and pipelines available out-of-the-box through NeMo Curator!
+
+We'll walk through an example of each pipeline, as well as how you could make modifications to the provided pipelines.
+
+> NOTE: Currently, the `convert_response_to_yaml_list()` method is extremely strict - manual parsing of the intermediate results is recommended in all cases. In the notebook we have wrapped these in `try/except` blocks to ensure you can move through the notebook without being impeded by the error.
+
+### Covered Tools:
+
+Through the following tools, NeMo Curator offers the following tools, which are compatible with both OpenAI API compatible models hosted on `build.nvidia.com`, as well as any LLM NIM that is locally running.
+
+- NeMo Curator OpenAI Client (Sync and Async)
+- Chat and Reward Model Usage
+
+### Covered Pipelines:
+
+Through the use of the `NemotronGenerator`, NeMo Curator offers the following pipelines:
+
+- Math Question Generation Pipeline
+- Writing Task Generation Pipeline
+- Open Question Generation Pipeline
+- Closed Question Generation Pipeline
+- Python Question Generation Pipeline
+- Dialogue Generation Pipeline
+- Two-Turn Prompt Generation Pipeline
+- Entity Classification
+    - Classify Math Entity
+    - Classify Python Entity
+
+> NOTE: If you are using the `build.nvidia.com` endpoint for Nemotron-4 340B Instruct as your model for the above pipelines, during times of high load, it's possible that pipelines might time-out. In this case, we would recommend running the pipeline in a piecewise fashion and saving the intermediate outputs.