You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the schema generator requires a full working environment within the docker environment in order to work. This means to test new changes to the schema generator, the entire container must be rebuilt at each specific revision.
If the schema generation logic in bin were moved into the main package, with only the git and jsonschema-transpiler dependencies required in the environment, then it would be much easier to test generated schemas.
The ideal workflow would involve calling the application within a schema repository. For example:
This would allow us to move away from coreutils and bash for schema generating logic. It is difficult (but not impossible) to test changes to the schema generator. However, it would be easier to test things when the script can be broken into smaller chunks.
We should make pushing to git an optional. Sometimes, what we want to do is to generate a diff between two versions of the schema repository. For this, we can specify an interim folder where generated schemas can be dumped into.
Currently, the schema generator requires a full working environment within the docker environment in order to work. This means to test new changes to the schema generator, the entire container must be rebuilt at each specific revision.
If the schema generation logic in bin were moved into the main package, with only the git and jsonschema-transpiler dependencies required in the environment, then it would be much easier to test generated schemas.
The ideal workflow would involve calling the application within a schema repository. For example:
cd mozilla-pipeline-schemas mozilla-schema-generator generate \ --input schemas/ \ --interim generated-schemas/ \ --output-branch test-generated-schemas
This would allow us to move away from coreutils and bash for schema generating logic. It is difficult (but not impossible) to test changes to the schema generator. However, it would be easier to test things when the script can be broken into smaller chunks.
We should make pushing to git an optional. Sometimes, what we want to do is to generate a diff between two versions of the schema repository. For this, we can specify an
interim
folder where generated schemas can be dumped into.Some of the utilities and testing from https://github.com/mozilla-services/mozilla-pipeline-schemas/blob/master/scripts/bigquery_schema_diff.py may be useful here. In particular, the
tmp_git
fixture would be useful for testing pushes togenerated-schemas
, without requiring an SSH key.The text was updated successfully, but these errors were encountered: