Build regex from JSON Schema in Rust

[ErikKaum]

Resolves #12

In relation to the conversation in #12 I opened this PR to share the code that I wrote a while ago. Essentially this was intended as a 1-to-1 Rust port of the build_regex_from_schema function.

Running pytest tests/fsm/test_json_schema.py I get: 18 failed, 45 passed, this excludes the tests for to_regex since that function isn't fully implemented on the rust side.

• write build_regex_from_schema and to_regex to rust
• get all tests to pass
• address all todos left in code
• remove all .unwrap()s
• clean up code & make more readable
• ensure performance hasn't been degraded

---

NB!
The original python code has a

    schema = json.loads(schema)
    Validator.check_schema(schema)

which I didn't find a good rust equivalent for. We can use the jsonschema crate like so:

    let compiled_schema = JSONSchema::compile(&json_value)
        .map_err(|e| anyhow!("Failed to compile JSON schema: {}", e))?;

but it's slightly different and using for example "float" data types fail.
Additionally there's already the let json_value: Value = serde_json::from_str(json)?; which performs a form of validation on the incoming json.

[brandonwillard]

This looks great, thanks!

[brandonwillard]

FYI: I just rebased this to account for the changes in main.

[ErikKaum]

Status update: all the test for the build_regex_from_schema function pass. The next step is exposing the to_regex function and making sure those tests pass as well. The test for to_regex have been commented out for now.

[ErikKaum]

All tests pass 🙌 I'll clean up the code, remove todos and unwraps and then we can proceed for review.

[ErikKaum]

Benchmark results

[ErikKaum]

Additional questions for reviewers. The original python implementation exposed the to_regex:

def to_regex(
    resolver: Resolver, instance: dict, whitespace_pattern: Optional[str] = None
):

But as far as I can tell it's only used in tests and only consumed by the build_regex_from_schema function.
Should we leave it unexposed to python and not have bindings to it?

Similarly I left function get_schema_from_signature to the python side since it's so python specific and probably wouldn't make sense to have it exposed to other languages. Maybe we even should move it to outlines completely?

[brandonwillard]

But as far as I can tell it's only used in tests and only consumed by the build_regex_from_schema function.
Should we leave it unexposed to python and not have bindings to it?

It can be a rather useful means of debugging (e.g. in an issue), so it's probably worth exposing at the Python level, but we can skip that for now if you want.

[ErikKaum]

It can be a rather useful means of debugging (e.g. in an issue), so it's probably worth exposing at the Python level, but we can skip that for now if you want.

Okay that's good to know. For me we can keep it, no worries 👍

[brandonwillard]

Had to rebase again for upstream fixes/changes.

Also, I'll provide a review shortly!

[brandonwillard]

Looks great!

[ErikKaum]

Thank you 🙌