Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better jinja error #8

Merged
merged 2 commits into from
May 15, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -9,6 +9,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

* Added config section in readme.
* Added examples.
* Added many tests.
* Added `raise` jinja macro.
* Generated project with template.
* Added badges.
* Added contributors.
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -11,6 +11,22 @@

</div>

## Config

The config is comprised of a single file. (Currently there is no fixed filename.)
For ease of use you should use the file extension `.yml.jinja` and develope said config file as two seperate files at first.
The config file has two parts delimited by `------` (6 or more `-`).
The first part is the schema (*What is valid user input?*). The second part is the template (*What is the actual command for said user input?*).
The first part must be a valid [JSON Schema](https://json-schema.org/) in [JSON](https://www.json.org/json-en.html) or to JSON capabilites restricted [YAML](https://yaml.org/) and the second part must be a valid [jinja2 template](https://jinja.palletsprojects.com/en/3.0.x/templates/).
The exception to this is that `anyOf` and `oneOf` are only usable for enum like behaviour on the schema side.
Instead of an `enum` you might want to use `anyOf` with `const` and `tile` properties.
The `title` of a property is used for the pretty print name, while the `description` is used for the help message.
There exists a jinja2 macro called `raise`, which is usefull if there is actually no installation instruction for said user input.
All lineends in the template are removed after render, which means that commands can be splitted within the template (`conda install {{ "xyz" if myvar else "abc" }}` ).
This also means that multiple commands need to be chained via `&&`.
For examples please look at the [examples folder](./examples/).


## Installation

The Python package `installation_instruction` can be installed from PyPI:
12 changes: 4 additions & 8 deletions examples/pytorch/pytorch-instruction.schema.yml.jinja
Original file line number Diff line number Diff line change
@@ -64,9 +64,7 @@ additionalProperties: false
{% if compute_platform == "cpu" %}
pytorch{{ "-nightly" if build == "preview" }}::pytorch torchvision torchaudio
{% else %}
[[ERROR]]
Mac does not support ROCm or CUDA!
[[ERROR]]
{{ raise("Mac does not support ROCm or CUDA!") }}
{% endif %}
{% else %}
pytorch torchvision torchaudio
@@ -76,9 +74,7 @@ additionalProperties: false
{% elif compute_platform == "cu121" %}
pytorch-cuda=12.1 -c nvidia
{% elif compute_platform == "ro60" %}
[[ERROR]]
ROCm is currently not supported with conda on linux and not supported at all on windows!
[[ERROR]]
{{ raise("ROCm is currently not supported with conda on linux and not supported at all on windows!") }}
{% else %}
cpuonly
{% endif %}
@@ -90,7 +86,7 @@ additionalProperties: false
pip3 install {{ "--pre" if build == "preview" }} torch torchvision torchaudio

{% if os == "mac" %}
{{ "[[ERROR]] Mac does not support ROCm or CUDA! [[ERROR]]" if compute_platform != "cpu" }}
{{ raise("Mac does not support ROCm or CUDA!") if compute_platform != "cpu" }}
{% if build == "preview" %}
--index-url https://download.pytorch.org/whl/nightly/cpu
{% endif %}
@@ -109,7 +105,7 @@ additionalProperties: false
{% endif %}

{% elif compute_platform == "ro60" %}
{{ "[[ERROR]] Windows does not support ROCm! [[ERROR]]" if os != "linux" }}
{{ raise("Windows does not support ROCm!") if os != "linux" }}
--index-url https://download.pytorch.org/whl/{{ "nightly/" if build == "preview" }}rocm6.0

{# CPU #}
2 changes: 1 addition & 1 deletion installation_instruction/helpers.py
Original file line number Diff line number Diff line change
@@ -25,7 +25,7 @@ def _get_error_message_from_string(string: str) -> str | None:
:return: Error message if found else None.
:rtpye: str or None
"""
reg = re.compile(r".*\[\[ERROR\]\]\s*(?P<errmsg>.*?)\s*\[\[ERROR\]\].*", re.S)
reg = re.compile(r"^.*\'\[ERROR\]\s*(?P<errmsg>.*?)\s*\'.*$", re.S)
matches = reg.search(string)
if matches is None:
return None
28 changes: 17 additions & 11 deletions installation_instruction/installation_instruction.py
Original file line number Diff line number Diff line change
@@ -14,17 +14,21 @@


from yaml import safe_load

import json

from jsonschema import validate

from jinja2 import Environment, Template

from jinja2.exceptions import UndefinedError

import installation_instruction.helpers as helpers


RAISE_JINJA_MACRO_STRING = """
{% macro raise(error) %}
{{ None['[ERROR] ' ~ error][0] }}
{% endmacro %}
"""


class InstallationInstruction:
"""
Class holding schema and template for validating and rendering installation instruction.
@@ -34,7 +38,7 @@ def validate_and_render(self, input: dict) -> tuple[str, bool]:
"""
Validates user input against schema and renders with the template.
Returns installation instructions and False.
If template '[[ERROR]]' is called returns error message and True.
If jinja macro `raise` is called returns error message and True.

:param input: Enduser input.
:ptype input: dict
@@ -43,10 +47,12 @@ def validate_and_render(self, input: dict) -> tuple[str, bool]:
:raise Exception: If schema or user input is invalid.
"""
validate(input, self.schema)
instruction = self.template.render(input)

if error := helpers._get_error_message_from_string(instruction):
return (error, True)

try:
instruction = self.template.render(input)
except UndefinedError as e:
if errmsg := helpers._get_error_message_from_string(str(e)):
return (errmsg, True)

instruction = helpers._replace_whitespace_in_string(instruction)

@@ -55,7 +61,7 @@ def validate_and_render(self, input: dict) -> tuple[str, bool]:

def __init__(self, config: str) -> None:
"""
Returns `InstallationInstruction` from config string.
Returns `InstallationInstruction` from config string. This also adds raise macro to template.

:param config: Config string with schema and template seperated by delimiter.
:raise Exception: If schema part of config is neither valid json nor valid yaml.
@@ -70,7 +76,7 @@ def __init__(self, config: str) -> None:
except:
raise Exception("Schema is neither a valid json nor a valid yaml.")

self.template = helpers._load_template_from_string(template)
self.template = helpers._load_template_from_string(RAISE_JINJA_MACRO_STRING+template)


def from_file(path: str):
24 changes: 21 additions & 3 deletions tests/test_helpers.py
Original file line number Diff line number Diff line change
@@ -15,14 +15,32 @@

from jinja2 import Template
from installation_instruction import helpers
from installation_instruction.installation_instruction import InstallationInstruction


def test_get_error_message_from_string_with_macro():
example_config = r"""
type: object
properties:
err:
type: boolean
------
something
{{ raise("test message") if err }}
something
"""

install = InstallationInstruction(example_config)

assert install.validate_and_render({ "err": True }) == ("test message", True)
assert install.validate_and_render({ "err": False }) == ("something something", False)



def test_get_error_message_from_string_with_err_message():
err_string = """
abcd
[[ERROR]]
Mac does not support ROCm or CUDA!
[[ERROR]]
'[ERROR] Mac does not support ROCm or CUDA!'

efg
"""