Skip to content

Commit

Permalink
Claryfying the scoring explanation & Refactoring
Browse files Browse the repository at this point in the history
of the file structure
  • Loading branch information
laurensWe committed Apr 15, 2024
1 parent 94bd020 commit ae2551f
Show file tree
Hide file tree
Showing 8 changed files with 10 additions and 9 deletions.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ALTAI

See the [introduction](tools.md#ai-assessment-tool). It is a discussion tool about AI Systems.
See the [introduction](../Comparison/tools.md#ai-assessment-tool). It is a discussion tool about AI Systems.

## Functionality

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI Verify

See the [introduction](tools.md#ai-verify)
See the [introduction](../Comparison/tools.md#ai-verify)

## Functionality

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Holisticai

See the [introduction](tools.md#holisticai). It is a toolkit just like IBM-360-Toolkit for a
See the [introduction](../Comparison/tools.md#holisticai). It is a toolkit just like IBM-360-Toolkit for a
data scientist to research bias and also to mitigate it immediately.

## Functionality
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# IBM Research 360 Toolkit

See the [introduction](tools.md#ibm-research-360-toolkit), same thing as verifyML this has no
See the [introduction](../Comparison/tools.md#ibm-research-360-toolkit), same thing as verifyML this has no
frontend baked in, but has some nice integrations with MLops tooling like Kubeflow
Pipelines. The IBM Research 360 toolkit is actuallt a collection of three open-source
toolkits as stated by their [Github repo](https://github.com/IBM/ai-360-toolkit-explained);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# VerifyML

See the [introduction](tools.md#verifyml), the maker also suggests
See the [introduction](../Comparison/tools.md#verifyml), the maker also suggests
to use an front-end tool to collaboratively change the model card.
[Model Card Editor](https://report.verifyml.com/) this is not
open-source and also the developer suggests in this
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,14 @@
[We have researched](tools.md) a few tools which we want to investigate
further, this document is the next step in that investigation.
We created a [checklist](index.md) to compare these tools against.
The Fulfilled column will give a numerical value based
on the priority which can then be compared with the other tools per section.
The scoring is as follows: {M:4, S:3, C:2, W:-1}.
The Fulfilled column will give a numerical value based on whether that
requirement is fulfilled or not between 0 and 1. Then the actual scoring
is the fulfilled value times the priority (the priority is translated
to numerical values in the following way: {M:4, S:3, C:2, W:-1}).

## Summary of the comparison

| Requirement | [AIVerify](aiverify_checklist.md) | [VerifyML](verifyml_checklist.md) | [IBM 360 Research Toolkit](ibm_360_research_toolkit_checklist.md) | [Holisticai](holisticai_checklist.md) | [AI Assessment Tool](ai_assessment_tool_checklist.md) |
| Requirement | [AIVerify](../Checklists/aiverify_checklist.md) | [VerifyML](../Checklists/verifyml_checklist.md) | [IBM 360 Research Toolkit](../Checklists/ibm_360_research_toolkit_checklist.md) | [Holisticai](../Checklists/holisticai_checklist.md) | [AI Assessment Tool](ai_assessment_tool_checklist.md) |
|:-----------------------|:----------------------------------|------------------------------------|--------------------------------------------------------------------|---------------------------------------|-------------------------------------------------------|
| Functionality | 36 | 42 | 20 | 17 | 22.85 |
| Reliability | 13 | 4 | 16 | 16 | 15.4 |
Expand Down

0 comments on commit ae2551f

Please sign in to comment.