Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test results for WholeTale v0.8.0rc1 (Chrome) #92

Closed
20 of 40 tasks
craig-willis opened this issue Sep 18, 2019 · 0 comments
Closed
20 of 40 tasks

Test results for WholeTale v0.8.0rc1 (Chrome) #92

craig-willis opened this issue Sep 18, 2019 · 0 comments

Comments

@craig-willis
Copy link
Collaborator

craig-willis commented Sep 18, 2019 •
edited
Loading

Test Plan

Note: For all tests, repeat for supported browser/OS combinations.

Preconditions:

  • Clear browsing history or open incognito/private session

Authentication

  • Basic login flow
  1. Goto https://dashboard.wholetale.org
  2. Select "Access Whole Tale" button. You should be prompted to login via Globus Auth.
  3. Select your organization and login
  4. You should be taken to the Browse page
  5. Your Gravatar and user name should display
  • Basic logout flow
  1. Select the "Logout" button. You should be taken to the login page.
  2. Select "Access Whole Tale" button.
  3. Note that you are not prompted to login.
    1. Note PR #331 for issues with logout
    2. Note issue #384 for logout for Globus
  4. Restart your incognito session
  5. Select "Access Whole Tale" button. You should be prompted to login via Globus Auth.
  • Return-route for non-logged in users (#300)
  1. Logout from Whole Tale
  2. Attempt to access https://dashboard.dev.wholetale.org/manage
  3. You should be redirected to https://dashboard.dev.wholetale.org/login?rd=%252Fmanage
  4. Select "Access Whole Tale".
  5. You should be redirected to the Manage page.

Navigation

  • General navigation
  1. Select "Browse" button. Browse page should display
  2. Select "Manage" button. Manage page should display
  3. Select "i" button should open User Guide in new tab
  4. Select "Logout" button should logout from WT
  5. Select "Report a problem" button should open https://github.com/whole-tale/whole-tale/issues in new tab
  6. Select "View" on a Tale. The Run page should display

Browse

Preconditions:

  • Assumes production Tales present (e.g., LIGO, materials, etc).

  • No running instances

  • General

  1. Select "Browse" button. Confirm browse page displays
  2. Confirm "i" button opens https://wholetale.readthedocs.io/en/stable/users_guide/browse.html
  3. Confirm "fullscreen" button opens Browse panel in fullscreen
  4. Confirm that no Tales are running
  • Search
  1. Enter "Ligo" as search term. Confirm only LIGO tale displays
  2. Clear search term. Confirm all tales display
  • Filter
  1. Navigate to the Browse page
  2. Confirm that you can switch between "All Tales" and "My Tales"
  3. Confirm that all public Tales are shown on "All Tales"
  4. Confirm that your Tales are shown under "My Tales"
  5. Select "My Tales"
  6. View a Tale and then navigate back to Browse
  7. Confirm the filter is reset
  • View tale
  1. Select "View" button on LIGO Tale
  2. Confirm View page displays
  3. Confirm that you cannot delete or edit Tale properties
  4. Select Back button. Confirm that you are taken back to Browse page
  5. Select "View" button on Tale you own
    1. Confirm that the Run and Close buttons are present
    2. Confirm that you can edit Tale properties
    3. Select "Close"
    4. Confirm that you are taken back to Browse
  • Launch instance
  1. Select "Run Tale" button on "Ligo" tale
    1. Confirm you are prompted to Copy and Run
    2. Select Copy and Run
    3. Confirm that you are redirected to the Tale's Run page
    4. Confirm that the notification dialog appears
    5. Confirm that the step progression is correct
  2. Select "Stop" to end the Tale instance
    1. Confirm that the Tale has stopped
    2. Confirm that the button has changed to Run
  3. Navigate to Browse and select tje "Run Tale" button on a Tale you own
    1. After the instance is launched, try to delete the Tale (not instance)
    2. Confirm you cannot select the delete button on the browse page
    3. Confirm that the "Run Tale" button has been replaced with "Stop Tale"

Manage

Preconditions

  • Empty home directory

  • No registered data

  • Register General

  1. Select Manage > Data > "+" to open register panel
  2. Confirm list of providers matches list of supported providers (Globus, DataONE, Dataverse)
  • Register DataONE data
  1. Select Manage > Data tab
  2. Select "+". Confirm "Search for Data Sources" modal displays
  3. Search for DOI 10.5065/D6862DM8
  4. Confirm "Humans and Hydrology" dataset is found on DataONE.
  5. Select Register
  6. Confirm dataset appears in Manage Data list
  7. Confirm additional URL formats:
    1. https://search.dataone.org/view/doi:10.5063/F1Z60M87
    2. https://search.dataone.org/view/https://pasta.lternet.edu/package/metadata/eml/knb-lter-nwt/115/1
    3. https://search.dataone.org/view/doi:10.18739/A2CN6XZ7H
    4. https://raw.githubusercontent.com/whole-tale/dashboard/master/.travis.yml
  • Dataverse
  1. Register dataset doi:10.7910/DVN/SZ9YXZ
  2. Confirm "Replication Data for: Direct Democracy and Women's Political Engagement" from Dataverse
  • Globus/MDF
  1. Confirm DOI resolution for http://dx.doi.org/doi:10.18126/M2301J results in https://publish.globus.org/jspui/handle/ITEM/113
  2. Register dataset https://publish.globus.org/jspui/handle/ITEM/113

Tale Creation

  • Create RStudio Tale
  1. Select the Create New... button on the Browse Page
  2. Select the Create New Tale dropdown item
  3. Enter title "Test Tale"
  4. Select compute environment RStudio
  5. Select "Create New Tale and Launch"
  6. Confirm notification dialog opens
  7. Confirm that the build/launch steps are correct
  8. Confirm that you are redirected to the Run page

  • Compose Jupyter Tale

  • Compose OpenRefine Tale

  • Too many instances

  1. Make sure you have two launched Tales
  2. Attempt to create a Tale
  3. Confirm that you receive an error that you have the max limit
  4. Confirm that the Tale is created, but not the instance

Run

Preconditions:

  • No running Tale instances

  • General

  1. Launch the Ligo Tale and wait for instance to start
  2. Confirm you are on the Run page
    1. Confirm Run panel displays Tale icon, title, and author
    2. Confirm Run page displays Interact, Files, Metadata tabs
  3. Select "..." menu.
    1. Read the docs displays https://wholetale.readthedocs.io/en/stable/users_guide/run.html
    2. Confirm fullscreen displays Run panel in full screen
    3. Confirm that the options for exporting the Tale are present
  • Interact tab
  1. Select the "Interact" tab
  2. Confirm Jupyter environment is displayed in iframe
  3. Select "popout" icon. Confirm Jupyter is opened in new Tab
  • Files tab
  1. Select "Files" tab
  2. You should see a vertical navigation menu with three tabs: "Home", "External Data", and "Tale Workspace".
  • Metadata tab - Read Only Tale
  1. View (not launch) the LIGO Tale
  2. Navigate to Run
  3. Select "Metadata" tab
  4. Confirm that metadata displays for current tale
  5. Confirm that the form elements are read only
  • Metadata tab - Owned Tale
  1. Launch a Tale that you own
  2. Add multiple datasets that have DOIs to the Tale
  3. Navigate to Run > metadata
  4. Valid environments should be listed in the Environment dropdown menu
  5. Valid licenses should be listed in the License dropdown
  6. Confirm that you see Created by <your_name> under Authors
  7. Confirm that your Tale has the CC4 license assigned to it
  8. Confirm that the Tale's Published Location reads This Tale has not been published
  9. Confirm that there are citations for the external datasets
  10. Change the name of the Tale, Category, Description, Illustration, Public, License, and add multiple authors
  11. Select Save. Confirm changes are saved.
  12. Refresh the page. Your changes should be preserved between page refreshes, confirming that they are being properly persisted.
  13. Confirm that the new Tale authors are displayed on the Tale card
  • Home
  1. Select Files > Home. Confirm that your home directory displays
  2. Create a folder and upload a file to the folder
  3. Confirm file operations:
    1. Copy, rename, remove, download
  • External data
  1. Precondition: Register one or more Datasets (e.g. doi:10.18739/A29G5GD0V) on Manage > Data (see Manage > Register test cases above)
    • NOTE: You may need to wait for registration to complete
  2. Confirm "External Data" list is initially empty
  3. Confirm (+) button ("Select Data modal") behavior
    1. Confirm that "WholeTale Catalog" shows all registered datasets
    2. Confirm that "My Data" filters to only show your registered datasets
    3. "Add" data to associate it with a Tale
    4. "Remove" data to dissociate it with a Tale
    5. "Select" saves your changes back to the server
    6. "Cancel" aborts the operation without saving and reverts your changes
  4. Confirm "External Data" list matches what was saved
  • Workspaces
  1. Confirm create folder
  2. Confirm rename folder/file
  3. Confirm upload file
  4. Confirm copy file/folder from/to Home
  5. Confirm move file/folder from/to Home
  6. Confirm remove file/folder
  7. Select "+" button has options "New folder", "Upload file", "Select Data"
  8. Folder operations
    1. Confirm context menu Move To, Rename, Copy to Home, Download, Remove
    2. Move To...
      1. Issue -- should this be enabled
    3. Confirm rename folder
    4. Copy to Home confirm folder copied to home
    5. Confirm download -- zip file
    6. Confirm remove
  9. File operations
    1. Upload a file
    2. Confirm context menu Move To, Rename, Copy to Home, Download, Remove
    3. Move To... Home?
      1. Issue -- doesn't work
    4. Confirm rename folder
    5. Copy to Home confirm folder copied to home
    6. Confirm download
    7. Confirm remove
  10. Select "Select Data..."
    1. Note vertical navigation with Home and Tale Workspaces
    2. Copy to Workspace from Home
    3. Move to Workspace from Home
    4. Copy to Workspace from another workspace
    5. Move to workspace from another workspace
  • Exporting - ZIP
  1. Launch a Tale that you own
  2. Navigate to Run
  3. Click the ... dropdown
  4. Select Export as Zip
  5. Confirm that you are asked to start a download for the archive
  6. Open the zip archive and confirm that the top level has
    1. LICENSE
    2. README.md
    3. metadata/
  7. Navigate into the metadata folder
  8. Confirm that there is a manifest.json and environment.json file
  • Exporting - BagIt
  1. Launch a Tale that you own
  2. Navigate to Run
  3. Click the three-dot-dropdown
  4. Select 'Export as BagIt'
  5. Confirm that you are asked to start a download for the archive
  6. Open the archive
  7. Confirm that the top level has
    1. run-local.sh
    2. tagmanifest-sha256.txt
    3. tagmanifest-md5.txt
    4. manifest-sha256.txt
    5. manifest-md5.txt
    6. bag-info.txt
    7. README.md
    8. bagit.txt
    9. fetch.txt
    10. data/
    11. metadata/
  8. Navigate into the metadata folder
  9. Confirm that there is a manifest.json and environment.json file
  10. Navigate to /data
  11. Confirm that there is a LICENSE file

Analyze in Whole Tale

Task 1: Importing a Dataset from Dataverse

These test cases cover potential situations that can occur when importing datasets from Dataverse.

  • Case 1: Ideal Scenario:

Testing Steps:

  1. Remove all running Tale instances
  2. Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataverse?fileId=3323458&siteUrl=https%3A%2F%2Fdataverse.harvard.edu
  3. Confirm that the Tale title reads as Replication Data for: "Agricultural Fires and Health at Birth"
  4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
  5. Confirm that no environment is selected
  6. Confirm that the Create New Tale and Launch button is disabled
  7. Select an environment
  8. Click Create New Tale and Launch
  9. Confirm that you are redirected to the run page
  10. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
  11. Confirm that the data exists in the Tale

Task 2: Importing a Dataset from DataONE

These test cases cover different scenarios that may happen when importing datasets from DataONE.

  • Case 1: Ideal Scenario:

This test covers the ideal case where dataset importing has all of the needed information
to properly run. This includes the dataset location, title, and selected environment.

Testing Steps:

  1. Remove all running Tale instances
  2. Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataone?uri=https%3A%2F%2Fsearch.dataone.org%2Fview%2Fdoi%3A10.18739%2FA2VQ2S94D&title=Fire%20influences%20on%20forest%20recovery%20and%20associated%20climate%20feedbacks%20in%20Siberian%20Larch%20Forests%2C%20Russia&environment=RStudio
  3. Confirm that the Tale title reads as Fire influences on forest recovery and associated climate feedbacks in Siberian Larch Forests, Russia
  4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
  5. Confirm that Rstudio is selected in the Environments widget
  6. Confirm that the Create New Tale and Launch button is enabled
  7. Click Create New Tale and Launch
  8. Confirm that you are redirected to the run page
  9. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
  10. Confirm that the data exists in the Tale

Task 3: Shared Behavior

These cases show errors that are common between Dataverse and DataONE. Although they are tested using DataONE URIs, the errors should be the same when replacing it with a Dataverse URI.

  • Case 1: Missing Data Location Query Param:

The test covers the case where the dataset location is missing as a query parameter. This query parameter is used to let the UI know
that we're importing a dataset/Tale. If it is missing, the default Compose page should be observed.

Testing Steps:

  1. Remove all running Tale instances
  2. Navigate to https://dashboard.stage.wholetale.org/compose?uri=
  3. Confirm that the Tale title is blank
  4. Confirm that the Selected data section is blank
  5. Confirm that no environment is selected
  6. Confirm that the Create New Tale and Launch button is disabled
  • Case 2: Missing Dataset Title Query Param:

This test covers the case where the name query parameter is missing. The expected behavior is
that the uri query parameter is used instead.

Testing Steps:

  1. Remove all running Tale instances
  2. Navigate to https://dashboard.stage.wholetale.org/compose?uri=https%3A%2F%2Fdataverse.harvard.edu%2Fapi%2Faccess%2Fdatafile%2F3323458
  3. Confirm that the Tale title reads as https://dataverse.harvard.edu/api/access/datafile/3323458
  4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
  5. Confirm that no Environment is selected
  6. Confirm that the Create New Tale and Launch button is disabled
  7. Select an environment
  8. Click Create New Tale and Launch
  9. Confirm that the notification bar appears & properly updates
  10. Confirm that you are redirected to the run page
  11. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
  12. Confirm that the data exists in the Tale

Tale metadata tests

The purpose of these tests are to confirm that the metadata files (manifest.json, environment.json, LICENSE) we generate are correct.

  • manifest.json
  1. Launch a Tale that you own
  2. Add a dataset to the Tale
  3. Add at least one author
  4. Add at least one file to the workspace
  5. Export the Tale
  6. Open the manifest.json file under metadata/
  7. Confirm that the author is listed under schema:author
  8. Confirm that you are listed under createdBy
  9. Confirm that the file(s) in the workspace are properly listed in aggregates
  10. Confirm that the external dataset is referenced in aggregates
  11. Confirm that the dataset is also listed under Datasets
  • environment.json
  1. Open the archive from the manifest.json test
  2. Navigate to metadata/environment.json
  3. Open the girder web API
  4. Locate and GET the Image that the Tale used
  5. Confirm that the config section from girder matches the config section in environment.json
  • LICENSE
  1. Open the archive from the manifest.json test
  2. Open the LICENSE file
  3. Navigate to Run > metadata
  4. Confirm that the opened LICENSE file matches the selected license on the metadata page
  5. Change the Tale's license
  6. Export the Tale
  7. Confirm that the LICENSE changed appropriately

Publishing tests

  • Case 1: Un-published Tale

    1. Launch a Tale that you own
    2. Add data to your Tale-this can be to the workspace or as external data
    3. Click the ellipsis dropdown on the Run page
    4. Click Publish tale...
    5. Confirm that you are presented with a popup asking you to login to DataONE
    6. Click Continue to DataONE Login
    7. Log into ORCID
    8. Confirm you are redirected back to Whole Tale
    9. Confirm the publishing modal automatically opens
    10. Confirm the the links in the info icons open in new tabs and resolve
    11. Confirm that you can select production and test DataONE servers
    12. Select the DataONE Development repository
    13. Click Publish
    14. Confirm that the progress bar begins and disappears once complete
    15. Confirm that an identifier is shown in the modal dialog.
    16. Click the identifier
    17. Confirm that you are brought to DataONE in a new tab
    18. Confirm that the package lists
      1. metadata.xml
      2. LICENSE
      3. README.md
      4. manifest.json
      5. environment.json
    19. Confirm that the Data Set Creators section lists your account
    20. Confirm that the Data Set Usage Rights matches the license on the Tale's Run > metadata page
    21. Confirm that the Published Location on the Run > metadata page is correct

  • Case 2: Re-published Tale

  1. Launch a Tale that has been published before
  2. Take note of the Published Location in Run > metadata
  3. Re-publish the Tale to the DataONE Development repository
  4. Confirm that the Published Location has changed to the correct package landing page

Regression tests

  • Internal-state problems (#326)
  1. Open a non-incognito browser tab, log into the UI and navigate to the /manage view
  2. Create a folder (e.g. test123) and click down into it
  3. Open an incognito browser tab, login to the UI again, and navigate to the /manage view
  4. In the incognito tab, delete the test123 folder and close the incognito tab
  5. Refresh your non-incognito browser tab
  6. You should not be stuck on a blue screen
  7. You should not need to manually refresh the page
  8. You should be taken to the /manage view of your Home folder
  9. NOTE: Your last refresh may take a couple of seconds longer than usual
  • Invalid instance ID (#333)
  1. Manually change the path suffix in the address bar to /run/ThisIsObviouslyNotARealID
  2. You should be brought to /browse page instead
  • Files don't overflow Manage page (#410)
  1. Navigate to the manage page
  2. Register enough/upload enough data to the point where you need to scroll through files on the right hand side
  3. Make sure that the files don't overflow the container, also check in Firefox
  • Handle read only Tales (#406)
  1. Login to the WholeTale Dashboard
  2. Launch a Tale that does not allow you read access
  3. Navigate to Run > Files
  4. You should see that Tale Workspaces has been greyed out, disallowing reads
  5. Launch a Tale that allows you to read, but not write
  6. Navigate to Run > Files > Tale Workspaces
    1.You should see the Tale's Workspace folders/files are listed here
  7. You should see that (+) has been greyed out, disallowing writes
@ThomasThelen ThomasThelen mentioned this issue Jan 15, 2020
Closed
22 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@craig-willis