Run a first technical test

[anneschuth]

For now we assume a model to be present and data to be present and it runs locally, and that it is linked from the system card.

• the system card points to the model in the model card which has a URI to a model.
• The URI (for now) is a file path (file://) to the guusje model.
• We add a simple example model (as a fixture), simply checked into our code base (use model from https://github.com/MinBZK/example-data-science-project)
• The path to the model is hardcoded (for now)
• The path for the data is hardcoded (for now)

We add a Shap technical test to the Instrument Registry. It needs

• A urn
• A description
• A task of type "technical test"
• The task should require a model by pointing to the model field included in the system card.
• The task has a urn (used below)
• To define where the results should be written in the system card (in measures)

Implement the technical test that runs the Shap implementation of AI Verify using the model pointed to in the system card. We implement this as a CLI (for now, later this becomes a Celery worker), based on the minimimal ai verify repo (https://github.com/MinBZK/aiverify/tree/using_aiverify_as_api).

The implementation:

• itself knows what instrument task it implements (it knows the urn of the Shap technical test task, see above)
• has access to the instrument definition (by retrieving it using the urn)
• has access to a system card (receives the path as a CLI argument)
• calls the AI Verify API
• writes the result from the SHAP test into the model card on the path defined by the instrument.

Acceptance Criteria

• All shap values are stored in the system card -> model card -> model index -> results -> measures -> ...
• implement how to store shap values in model card without lose of data

[github-actions]

This issue did not have any activity in the last 90 days and will be removed after 30 days

[github-actions]

This issue did not have any activity in the last 90 days and will be removed after 30 days

[github-actions]

This issue is closed due to inactivity

[anneschuth]

:(

[anneschuth]

❤️