Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert request, response tests to a standard format for wider publication #60

Open
karenetheridge opened this issue Sep 24, 2023 · 5 comments

Comments

@karenetheridge
Copy link
Owner

This is modelled after the pattern used in JSON-Schema-Test-Suite.

each test group consists of:

  • description
  • comment (optional)
  • openapi specification document
  • tests: array
    • item: a test
      • description
      • comment (optional) - used for additional test explanation
      • serialized HTTP request (or response)
      • valid:
      • errors (optional): JSON Schema error responses, in basic format.
        • instanceLocation
        • keywordLocation
        • absoluteKeywordLocation (optional)
        • error

And of course we should provide a JSON Schema describing this test structure.

Convert all existing request and response tests to this format. If all details from the test are preserved, we can delete the originals.

@karenetheridge
Copy link
Owner Author

in the harness that makes use of this test format, run the tests twice, as we do now - once parsing into HTTP::Request/Response objects, and the other parsing into Mojo::Message::Request/Response objects. Perform any doctoring of the HTTP::Request/Response objects as needed (set missing protocol?)

@karenetheridge
Copy link
Owner Author

karenetheridge commented Sep 24, 2023

consider outputting the tests in yaml rather than json, for readability - I think yaml supports a "heredoc" operator?

use this as a guide: https://yaml-multiline.info/
I think I want to use request: |-8 to include the serialized request with indentation (for readability) that will be removed.

$; perl -MYAML=Load -MData::Dumper -wle'print Dumper(Load("---\nhello: foo\nbloop: |-\n    mary had a little lamb\n    its fleece as white as snow\n\n"))'
$VAR1 = {
          'bloop' => 'mary had a little lamb
its fleece as white as snow',
          'hello' => 'foo'
        };

The $UseBlock option in YAML can do this:

perl -MYAML=Dump -MData::Dumper -wle'$YAML::UseBlock = 1; print Dump({hello=>"foo",bar=>{bloop=>"foo\nbar\n\nblah"}})'
---
bar:
  bloop: |-
    foo
    bar
    
    blah
hello: foo

@karenetheridge
Copy link
Owner Author

We can autogenerate this by creating a harness which does some method modifications and then calls do ./t/validate_request.t:

  • add a wrapper around OpenAPI::Modern->new to push the openapi spec to the test list,
  • add a wrapper around the local request() sub to dump the serialized request for the individual test case,
  • add a wrapper around cmp_deeply to capture the expected (or actual) result and any errors, along with the test description string.

and then when execution is complete, dump the yaml version of the tests.

@karenetheridge
Copy link
Owner Author

and then we'll need to write a harness which knows how to read in the test cases and executes them, just as we do via Test::JSON::Schema::Acceptance. If this data format catches on we can publish the harness, and test cases, in Test::OpenAPI::Acceptance.

@karenetheridge
Copy link
Owner Author

kinda duped by #77

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant