Skip to content

Commit

Permalink
Intermediary results
Browse files Browse the repository at this point in the history
  • Loading branch information
@laurensWe
laurensWe committed Apr 10, 2024
1 parent 9d6fe76 commit f09b39e
Showing 1 changed file with 30 additions and 30 deletions.
60 changes: 30 additions & 30 deletions docs/Projects/TAD/ai_assesment_tool_checklist.md
View file
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
# ALTAI

Introduction is written here [tools.md](tools.md#ai-assessment-tool). Is a discussion tool about AI Systems.
Introduction is written here [tools.md](tools.md#ai-assessment-tool). It is a discussion tool about AI Systems.

## Functionality

| Requirement | Priority | Fulfilled | Comments |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------|:----------|:---------|
| The tool allows users to conduct technical tests on algorithms or models, including assessments of performance, bias, and fairness. To facilitate these tests, users can input relevant datasets, | M | | |
| The tool allows users to choose which tests to perform. | M | | |
| The tool allows users to fill out questionnaires to conduct impact assessments for AI. For example IAMA or ALTAI. | M | | |
| The tool can generate a human readable report. | M | | |
| The tools works with a standardized report format, that it can read, write, and update. | M | | |
| The tool supports plugin functionality so additional tests can be added easily. | S | | |
| The tool allows to create custom reports based on components. | S | | |
| It is possible to add custom components for reports. | S | | |
| The tool provides detailed logging, including tracking of different model versions, changes in impact assessments, and technical test results for individual runs. | S | | |
| The tool supports saving progress. | S | | |
| The tool can be used on an isolated system without an internet connection. | S | | |
| The tool offers options to discuss and document conversations. For example, to converse about technical tests or to collaborate on impact assessments. | C | | |
| The tool operates with complete data privacy; it does not share any data or logging information. | C | | |
| The tool allows extension of report formats functionality. | C | | |
| The tool can be integrated in a CI/CD flow. | C | | |
| The tool can be offered as a (cloud) service where no local installation is required. | C | | |
| It is possible to define and automate workflows for repetitive tasks. | C | | |
| The tool offers pre-built connectors or low-code/no-code integration options to simplify the integration process. | C | | |
| The tool offers options to discuss and document conversations. For example, to converse about technical tests or to collaborate on impact assessments. | C | | |
| Requirement | Priority | Fulfilled | Comments |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------|:----------|:----------------------------------------------------------------------------------------------------------------|
| The tool allows users to conduct technical tests on algorithms or models, including assessments of performance, bias, and fairness. To facilitate these tests, users can input relevant datasets, | M | 0 | The tool only allows for discussions not technical tests |
| The tool allows users to choose which tests to perform. | M | 0 | See above |
| The tool allows users to fill out questionnaires to conduct impact assessments for AI. For example IAMA or ALTAI. | M | 1 | This is very well supported by the tool |
| The tool can generate a human readable report. | M | 0.9 | There is an export functionality for the outcomes of the assessment, it offers a print dialog |
| The tools works with a standardized report format, that it can read, write, and update. | M | 0 | This report cannot be re-imported in a different tool as it only exports to pdf |
| The tool supports plugin functionality so additional tests can be added easily. | S | 0 | Not applicable |
| The tool allows to create custom reports based on components. | S | 0 | The report cannot be customised by the user |
| It is possible to add custom components for reports. | S | 0 | See above |
| The tool provides detailed logging, including tracking of different model versions, changes in impact assessments, and technical test results for individual runs. | S | 0.75 | There is even for the users an extensive audit trail what happened to assessment, not different model versions |
| The tool supports saving progress. | S | 1 | Yes this is supported |
| The tool can be used on an isolated system without an internet connection. | S | 1 | Yes it can be ran locally or in a docker container without internet |
| The tool offers options to discuss and document conversations. For example, to converse about technical tests or to collaborate on impact assessments. | C | 1 | This is the main feature of the tool |
| The tool operates with complete data privacy; it does not share any data or logging information. | C | 1 | Stored locally in a mongoDB |
| The tool allows extension of report formats functionality. | C | 0.5 | It could be developed that we export to markdown instead of pdf, but right now it just prints the window as pdf |
| The tool can be integrated in a CI/CD flow. | C | 0 | It is an UI tool, so doesn't make sense in a CI/CD pipeline |
| The tool can be offered as a (cloud) service where no local installation is required. | C | 1 | We could host this tool for other parties to use |
| It is possible to define and automate workflows for repetitive tasks. | C | 0 | It is an UI tool |
| The tool offers pre-built connectors or low-code/no-code integration options to simplify the integration process. | C | 0 | No |
| The tool offers options to discuss and document conversations. For example, to converse about technical tests or to collaborate on impact assessments. | C | | |

total_score =

Expand Down Expand Up @@ -91,14 +91,14 @@ total_score =

## Security

| Requirement | Priority | Fulfilled | Comments |
|:---------------------------------------------------------------------------------------------------------------------------------------|:---------|-----------|----------|
| The tool must protect data and system from unauthorized access, use, disclosure, disruption, modification, or destruction. | M | | |
| Regular security audits and penetration testing are conducted. | S | | |
| The tool enforce authorization controls based on user roles and permissions, restricting access to sensitive data and functionalities. | C | | |
| Data encryption is used for sensitive information at rest and in transit. | C | | |
| The project allows for regular security audits and penetration testing to identify vulnerabilities and ensure system integrity. | C | | |
| The tool implements backup functionality to ensure data availability in case of incidents. | C | | |
| Requirement | Priority | Fulfilled | Comments |
|:---------------------------------------------------------------------------------------------------------------------------------------|:---------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| The tool must protect data and system from unauthorized access, use, disclosure, disruption, modification, or destruction. | M | | |
| Regular security audits and penetration testing are conducted. | S | 0 | When running docker compose up, the docker client will tell there are quite some CVE vulnerabilities in there, an upgrade of the Node version would help much here |
| The tool enforce authorization controls based on user roles and permissions, restricting access to sensitive data and functionalities. | C | 1 | The tool |
| Data encryption is used for sensitive information at rest and in transit. | C | 1 | |
| The project allows for regular security audits and penetration testing to identify vulnerabilities and ensure system integrity. | C | 1 | The tool does allow this, as it is open-source |
| The tool implements backup functionality to ensure data availability in case of incidents. | C | | |

total_score =

Expand Down

0 comments on commit f09b39e

Please sign in to comment.