Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem: It is difficult to verify sample-data that cannot be modified where AMAUAT needs it #851

Open
ross-spencer opened this issue Aug 7, 2019 · 1 comment
Labels
AMAUAT Issues relating to the improvement of the AM Automated Acceptance tests

Comments

@ross-spencer
Copy link
Contributor

Please describe the problem you'd like to be solved.

Noted in this PR where we need some flexibility to modify and add to the sample data repository, there is a slight chance the change will impact AMAUAT testing, where aside from a small number of potential tests, we do need a certain amount of control data to verify the behavior of the system.

Describe the solution you'd like to see implemented.

Audit the range of transfers used across AMAUAT tests, e.g. this Bag Transfer used for aip-encrpyt-mirror, and move it, or create new AMAUAT samples in the acceptance-tests folder in the same repository.

It would be good to do this change in 1.11 early on in the release, or sometime shortly before testing. We have a much broader PR in-progress right now, that is close to merging, and it would just take a small amount of effort on a number of fronts to verify the efficacy of any changes to sample data at this point.

Describe alternatives you've considered.

Aside from the slight chance of minor duplication, e.g. two similar bags in two parts of the sample-data repository. We find benefits when we're free to change an manipulate the sample data used in more regular day-to-day/regression tests to make it more accurate/reliable to every day circumstances (or just more fun!). Similarly with AMAUAT tests.

Should we make this change we might also want to update AMAUAT process documentation and seek to ask developers to create new test data in this folder specifically.

Additional context

Related to: artefactual-labs/archivematica-acceptance-tests#72
Adjacent to: #797


For Artefactual use:
Please make sure these steps are taken before moving this issue from Review to Done:

  • All PRs related to this issue are properly linked 👍
  • All PRs related to this issue have been merged 👍
  • Test plan for this issue has been implemented and passed 👍
  • Documentation regarding this issue has been written and it has been added to the release notes, if needed 👍
@ross-spencer ross-spencer added AMAUAT Issues relating to the improvement of the AM Automated Acceptance tests triage-release-1.11 labels Aug 7, 2019
@ross-spencer
Copy link
Contributor Author

Folders and a list of transfers that should be used by sample data for AMAUAT:

uuids-for-dirs

* ~/archivematica-sampledata/TestTransfers/acceptance-tests/pid-binding/hierarchy-with-empty-dir
* TestTransfers/acceptance-tests/pid-binding/hierarchy-with-empty-dir
* ~/archivematica-sampledata/SampleTransfers/BagTransfer.zip
* SampleTransfers/BagTransfer.zip

transfer-policy-check

* TestTransfers/acceptance-tests/preforma/all-conform-policy-originals  | NYULibraries_MKVFFV1-MODIFIED.xsl
* TestTransfers/acceptance-tests/preforma/none-conform-policy-originals | NYULibraries_MKVFFV1-MODIFIED.xsl

transfer-mkv-conformance

* TestTransfers/acceptance-tests/preforma/all-valid
* TestTransfers/acceptance-tests/preforma/none-valid

premis-events-feature

* ~/archivematica-sampledata/SampleTransfers/BagTransfer
* ~/archivematica-sampledata/TestTransfers/Unicode
* ~/archivematica-sampledata/SampleTransfers/BagTransfer
* ~/archivematica-sampledata/SampleTransfers/BagTransfer
* ~/archivematica-sampledata/SampleTransfers/BagTransfer

pid-binding-feature

* TestTransfers/acceptance-tests/pid-binding/hierarchy-with-empty-dir | dir2/dir2a/dir2aiii |

performance-stdout-no-write

* ~/archivematica-sampledata/TestTransfers/small
* ~/archivematica-sampledata/SampleTransfers/Images
* ~/archivematica-sampledata/TestTransfers/acceptance-tests/performance/images-17M-each
* ~/archivematica-sampledata/TestTransfers/acceptance-tests/performance/video-14M-each

metadata-only-aip-reingest

* SampleTransfers/Images/pictures

mediaconch-challenging-mkv

* ~/archivematica-sampledata/mc-challenging-file

manual-normalization

* ~/archivematica-sampledata/TestTransfers/acceptance-tests/manual-normalization

ingest-policy-check

* TestTransfers/acceptance-tests/preforma/all-conform-policy-norm-acc   | NYULib_MKVFFV1_MODIFIED.xml
* TestTransfers/acceptance-tests/preforma/none-conform-policy-norm-acc  | NYULib_MKVFFV1_MODIFIED.xml
* TestTransfers/acceptance-tests/preforma/all-conform-policy-norm-acc   | NYULibraries_MKVFFV1-MODIFIED.xsl
* TestTransfers/acceptance-tests/preforma/none-conform-policy-norm-acc  | NYULibraries_MKVFFV1-MODIFIED.xsl

* TestTransfers/acceptance-tests/preforma/all-conform-policy  | NYULib_MKVFFV1_MODIFIED.xml
* TestTransfers/acceptance-tests/preforma/none-conform-policy | NYULib_MKVFFV1_MODIFIED.xml
* TestTransfers/acceptance-tests/preforma/all-conform-policy  | NYULibraries_MKVFFV1-MODIFIED.xsl
* TestTransfers/acceptance-tests/preforma/none-conform-policy | NYULibraries_MKVFFV1-MODIFIED.xsl

* TestTransfers/acceptance-tests/preforma/manually-normalized-preservation-all-conform-policy   | NYULib_MKVFFV1_MODIFIED.xml
* TestTransfers/acceptance-tests/preforma/manually-normalized-preservation-none-conform-policy  | NYULib_MKVFFV1_MODIFIED.xml
* TestTransfers/acceptance-tests/preforma/manually-normalized-preservation-all-conform-policy   | NYULibraries_MKVFFV1-MODIFIED.xsl
* TestTransfers/acceptance-tests/preforma/manually-normalized-preservation-none-conform-policy  | NYULibraries_MKVFFV1-MODIFIED.xsl

* TestTransfers/acceptance-tests/preforma/manually-normalized-access-all-conform-policy   | NYULib_MKVFFV1_MODIFIED.xml
* TestTransfers/acceptance-tests/preforma/manually-normalized-access-none-conform-policy  | NYULib_MKVFFV1_MODIFIED.xml
* TestTransfers/acceptance-tests/preforma/manually-normalized-access-all-conform-policy   | NYULibraries_MKVFFV1-MODIFIED.xsl
* TestTransfers/acceptance-tests/preforma/manually-normalized-access-none-conform-policy  | NYULibraries_MKVFFV1-MODIFIED.xsl

ingest-mkv-conformance

* TestTransfers/acceptance-tests/preforma/when-normalized-all-valid
* TestTransfers/acceptance-tests/preforma/when-normalized-none-valid

* TestTransfers/acceptance-tests/preforma/when-normalized-access-all-valid

indexless-feature

* ~/archivematica-sampledata/SampleTransfers/Images/pictures
* ~/archivematica-sampledata/SampleTransfers/BagTransfer

aip-encryption-mirror

* SampleTransfers/BagTransfer

aip-encryption

* SampleTransfers/BagTransfer

capture-output

* ???

black-box-checksum

* "SampleTransfers/DemoTransferCSV"
* "TestTransfers/fixityCheckShouldFail"

black-box-create-aip

* "SampleTransfers/DemoTransferCSV"
* "SampleTransfers/BagTransfer"
* "SampleTransfers/Dataverse/NDSAStaffingReport"

black-box-rights

* "SampleTransfers/DemoTransferCSV"

black-box-extract-package

* "SampleTransfers/OfficeDocs"
* "TestTransfers/broken_package_format_types"

black-box-reingest

* "SampleTransfers/DemoTransferCSV"
* "SampleTransfers/UnzippedBag"

black-box-transfer-microservices

* SampleTransfers/DemoTransferCSV
* TestTransfers/badNames

black-box-virus-feature

* "SampleTransfers/DemoTransferCSV"
* "TestTransfers/virusTests"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AMAUAT Issues relating to the improvement of the AM Automated Acceptance tests
Projects
None yet
Development

No branches or pull requests

1 participant