Merge pull request #333 from guardrails-ai/barney

Barney

.gitignore

@@ -1,3 +1,4 @@
+docs/api_reference_markdown/
 openai_api_key.txt
 *__pycache__*
 data/*
@@ -24,4 +25,11 @@ coverage.xml
 test.db
 test.index
 htmlcov
-.python-version
+node_modules
+.docusaurus
+docs-build
+.vercel
+docusaurus/examples-toc.json
+.python-version
+static/docs
+docusaurus/static/docs

Makefile

@@ -62,6 +62,11 @@ dev:
 full:
 	poetry install --all-extras
 
+docs-gen:
+	poetry run python ./docs/pydocs/generate_pydocs.py
+	cp -r docs docs-build
+	poetry run nbdoc_build --force_all True --srcdir ./docs-build
+
 self-install:
 	pip install -e .
 

docs/.gitignore

@@ -0,0 +1 @@
+/.quarto/

docs/concepts/envars.md

@@ -2,11 +2,11 @@
 
 Guardrails recognize a handful of environment variables that can be used at runtime.  Most of these correlate to envinronment variables used or expected by the various LLM clients.  Below you can find a list of these and their uses.
 
-## OPENAI_API_KEY
+### `OPENAI_API_KEY`
 This environment variable can be used to set your api key credentials for Open AI models.  It will be used wherever Open AI is called if an api_key kwarg is not passed to `__call__` or `parse`.
 
-## GUARDRAILS_PROCESS_COUNT
+### `GUARDRAILS_PROCESS_COUNT`
 This environment variable can be used to set the process count for the multiprocessing executor.  The multiprocessing executor is used to run validations in parallel where possible.  To disable this behaviour and force synchronous validation, you can set this environment variable to `'1'`.  The default is `'10'`.
 
-## INSPIREDCO_API_KEY
+### `INSPIREDCO_API_KEY`
 This environment variable can be used to set your api key credentials for the Inspired Cognition API Client.  It will be used wherever the Inspired Cognition API is called.  Currently this is only used in the `is-high-quality-translation` validator.

docs/concepts/instructions.md

@@ -9,7 +9,7 @@ In addition to any static text describing the context of the task, instructions
 | Component         | Syntax                   | Description                                                                                                                                                                                                                                                                                                                             |
 |-------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | Variables         | `${variable_name}`        | These are provided by the user at runtime, and substituted in the instructions.                                                                                                                                                                                                                                                             |
-| Output Schema     | `${output_schema}`      | This is the schema of the expected output, and is compiled based on the  `output` element.  For more information on how the output schema is compiled for the instructions, check out [`output` element compilation](../output/#adding-compiled-output-element-to-prompt).                                                                    |
+| Output Schema     | `${output_schema}`      | This is the schema of the expected output, and is compiled based on the  `output` element.  For more information on how the output schema is compiled for the instructions, check out [`output` element compilation](/docs/concepts/output/#adding-compiled-output-element-to-prompt)                                                                    |
 | Prompt Primitives | `${gr.prompt_primitive_name}` | These are pre-constructed blocks of text that are useful for common tasks. E.g., some primitives may contain information that helps the LLM understand the output schema better.  To see the full list of prompt primitives, check out [`guardrails/constants.xml`](https://github.com/guardrails-ai/guardrails/blob/main/guardrails/constants.xml). |
 
 

docs/concepts/logs.md

@@ -1,10 +1,10 @@
-# Inspecting logs
+# Logs
 
 All `Guard` calls are logged internally, and can be accessed via the guard history.
 
 ## 🇻🇦 Accessing logs via `Guard.history`
 
-`history` is an attribute of the `Guard` class. It implements a standard `Stack` interface with a few extra helper methods and properties.  For more information on our `Stack` implementation see the [Helper Classes](/api_reference/helper_classes) page.
+`history` is an attribute of the `Guard` class. It implements a standard `Stack` interface with a few extra helper methods and properties.  For more information on our `Stack` implementation see the [Helper Classes](/docs/api_reference_markdown/helper_classes) page.
 
 Each entry in the history stack is a `Call` log which will contain information specific to a particular `Guard.__call__` or `Guard.parse` call in the order that they were executed within the current session.
 
@@ -188,4 +188,4 @@ completion token usage:  16
 token usage for this step:  633
 ```
 
-For more information on the properties available on `Iteration`, ee the [History & Logs](/api_reference/history_and_logs/#guardrails.classes.history.Iteration) page.
+For more information on the properties available on `Iteration`, see the [History & Logs](/docs/api_reference_markdown/history_and_logs/#guardrails.classes.history.Iteration) page.

docs/concepts/output.md

@@ -196,7 +196,7 @@ At the heart of the `RAIL` specification is the use of elements. Each element's
 
 Guardrails supports many data types, including:, `string`, `integer`, `float`, `bool`, `list`, `object`, `url`, `email` and many more.
 
-Check out the [RAIL Data Types](../data_types.md) page for a list of supported data types.
+Check out the [RAIL Data Types](/docs/api_reference_markdown/datatypes) page for a list of supported data types.
 
 
 #### Scalar vs Non-scalar types
@@ -207,7 +207,7 @@ Guardrails supports two types of data types: scalar and non-scalar.
 | Scalar                                                                  | Non Scalar                                                                           |
 |-------------------------------------------------------------------------|--------------------------------------------------------------------------------------|
 | Scalar types are void elements, and can't have any child elements.      | Non-scalar types can be non-void, and can have closing tags and child elements.                       |
-| Syntax:   ``` <string ... /> ```                                        | Syntax: <list ...>     <string /> </list>|
+| Syntax:   ``` <string ... /> ```                                        | Syntax: ```<list ...>     <string /> </list>```|
 | Examples: `string`, `integer`, `float`, `bool`, `url`, `email`, etc. | Examples: `list` and `object` are the only non-scalar types supported by Guardrails. |
 
 
@@ -217,20 +217,23 @@ Each element can have attributes that specify additional information about the d
 
 1. `name` attribute that specifies the name of the field. This will be the key in the output JSON. E.g.
 
-    === "RAIL Spec"
-        ```xml
-        <rail version="0.1">
-            <output>
-                <string name="some_key" />
-            </output>
-        </rail>
-        ```
-    === "Output JSON"
-        ```json
-        {
-            "some_key": "..."
-        }
-        ```
+=== "RAIL Spec"
+
+    ```xml
+    <rail version="0.1">
+        <output>
+            <string name="some_key" />
+        </output>
+    </rail>
+    ```
+
+=== "Output JSON"
+
+    ```json
+    {
+        "some_key": "..."
+    }
+    ```
 
 2. `description` attribute that specifies the description of the field. This is similar to a prompt that will be provided to the LLM. It can contain more context to help the LLM generate the correct output.
 3. (Coming soon!) `required` attribute that specifies whether the field is required or not. If the field is required, the LLM will be asked to generate the field until it is generated correctly. If the field is not required, the LLM will not be asked to generate the field if it is not generated correctly.
@@ -297,7 +300,7 @@ Each quality criteria is then checked against the generated output. If the quali
 ### Supported criteria
 
 - Each quality critera is relevant to a specific data type. For example, the `two-words` quality criteria is only relevant to strings, and the `positive` quality criteria is only relevant to integers and floats.
-- To see the full list of supported quality criteria, check out the [Validation](../api_reference/validators.md) page.
+- To see the full list of supported quality criteria, check out the [Validation](/docs/api_reference_markdown/validators) page.
 
 
 ## 🛠️ Specifying corrective actions

docs/concepts/prompt.md

@@ -9,7 +9,7 @@ In addition to the high level task description, the prompt also contains the fol
 | Component         | Syntax                   | Description                                                                                                                                                                                                                                                                                                                             |
 |-------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | Variables         | `${variable_name}`        | These are provided by the user at runtime, and substituted in the prompt.                                                                                                                                                                                                                                                               |
-| Output Schema     | `${output_schema}`      | This is the schema of the expected output, and is compiled based on the  `output` element.  For more information on how the output schema is compiled for the prompt, check out [`output` element compilation](../output/#adding-compiled-output-element-to-prompt).                                                                    |
+| Output Schema     | `${output_schema}`      | This is the schema of the expected output, and is compiled based on the  `output` element.  For more information on how the output schema is compiled for the prompt, check out [`output` element compilation](/docs/concepts/output/#adding-compiled-output-element-to-prompt).                                                                    |
 | Prompt Primitives | `${gr.prompt_primitive_name}` | These are pre-constructed prompts that are useful for common tasks. E.g., some primitives may contain information that helps the LLM understand the output schema better.  To see the full list of prompt primitives, check out [`guardrails/constants.xml`](https://github.com/guardrails-ai/guardrails/blob/main/guardrails/constants.xml). |
 
 ```xml

docs/concepts/validators.md

@@ -3,7 +3,7 @@
 Validators are how we apply quality controls to the schemas specified in our `RAIL` specs.  They specify the criteria to measure whether an output is valid, as well as what actions to take when an output does not meet those criteria.
 
 ## How do Validators work?
-When a validator is applied to a property on a schema, and output is provided for that schema, either by wrapping the LLM call or passing in the LLM output, the validators are executed against the values for the properties they were applied to.  If the value for the property passes the criteria defined, a `PassResult` is returned from the validator.  This `PassResult` tells Guardrails to treat the value as if it is valid.  In most cases this means returning that value for that property at the end; other advanced cases, like using a value override, will be covered in other sections.  If, however, the value for the property does not pass the criteria, a `FailResult` is returned.  This in turn tells Guardrails to take any corrective actions defined for the property and validation.  Corrective actions are defined by the `on-fail-...` attributes in a `RAIL` spec.  You can read more about what corrective actions are available [here](/concepts/output/#specifying-corrective-actions).
+When a validator is applied to a property on a schema, and output is provided for that schema, either by wrapping the LLM call or passing in the LLM output, the validators are executed against the values for the properties they were applied to.  If the value for the property passes the criteria defined, a `PassResult` is returned from the validator.  This `PassResult` tells Guardrails to treat the value as if it is valid.  In most cases this means returning that value for that property at the end; other advanced cases, like using a value override, will be covered in other sections.  If, however, the value for the property does not pass the criteria, a `FailResult` is returned.  This in turn tells Guardrails to take any corrective actions defined for the property and validation.  Corrective actions are defined by the `on-fail-...` attributes in a `RAIL` spec.  You can read more about what corrective actions are available [here](/docs/concepts/output#%EF%B8%8F-specifying-corrective-actions).
 
 ## Validator Structure
 ### Arguments
@@ -49,14 +49,14 @@ raw_output, guarded_output, *rest = guard(
 ```
 
 #### How do I know what metadata is required?
-First step is to check the docs.  Each validator has an API reference that documents both its initialization arguments and any required metadata that must be supplied at runtime.  Continuing with the example used above, `ExtractedSummarySentencesMatch` accepts an optional threshold argument which defaults to `0.7`; it also requires an entry in the metadata called `filepaths` which is an array of strings specifying which documents to use for the similarity comparison.  You can see an example of a Validator's metadata documentation [here](../api_reference/validators.md/#guardrails.validators.ExtractedSummarySentencesMatch).
+First step is to check the docs.  Each validator has an API reference that documents both its initialization arguments and any required metadata that must be supplied at runtime.  Continuing with the example used above, `ExtractedSummarySentencesMatch` accepts an optional threshold argument which defaults to `0.7`; it also requires an entry in the metadata called `filepaths` which is an array of strings specifying which documents to use for the similarity comparison.  You can see an example of a Validator's metadata documentation [here](/docs/api_reference_markdown/validators#extractedsummarysentencesmatch).
 
 Secondly, if a piece of metadata is required and not present, a `RuntimeError` will be raised.  For example, if the metadata requirements are not met for the above validator, an `RuntimeError` will be raised with the following message:
 
 > extracted-sentences-summary-match validator expects `filepaths` key in metadata
 
 ## Custom Validators
-If you need to perform a validation that is not currently supported by the [validators](../api_reference/validators.md) included in guardrails, you can create your own custom validators to be used in your local python environment.
+If you need to perform a validation that is not currently supported by the [validators](/docs/api_reference_markdown/validators) included in guardrails, you can create your own custom validators to be used in your local python environment.
 
 A custom validator can be as simple as a single function if you do not require addtional arguments:
 ```py