Basic Regression Testing

Surveys Tab

Project Structure

1. Create new folder called <release version> UAT Folder
2. Create three new folders within that one:
   • <release version> MF Surveys
   • <release version> Non MF Surveys
   • Test
3. Rename Test
4. Move Test to another folder
5. Delete Test

Monitored Surveys

1. Create a new survey within <release version> MF Surveys
2. Add a form called Registration Form
3. Add a free text question with "use as data point name" checked
4. Add a geolocation question with "use as data point location" checked
5. Add a number question with "use as data point name" checked
6. Publish
7. Go back to folder and open MF Survey again
8. Edit number question and un-check "use as data point name"
9. Save and make sure the setting changed
10. Publish
11. Scroll back up to top of the survey
12. Enable monitoring and ensure registration form is set
13. Add another form with some questions
14. Publish survey

Non Monitored Surveys

All questions

1. Create three new surveys within <release version> Non MF Surveys
2. Select one and name it <release version> All questions survey, change privacy setting from 'private' to 'public'
3. Add a form with one question group and then add a question for each question type
4. Make sure you create 2 types of option questions - one where options do not have 'codes' and one where they do have 'codes'
5. Make the group repeatable
6. Create a group with Caddisfly questions. One for EC and at least one for strip tests.
7. Select 'manage translations' from the form menu
8. Add any language, complete translations for all sections and save
9. Publish
10. Refresh the page (with F5 for example). Go back to the folder, and then select the <release version> All questions survey again.
11. Open the questions and make sure all question settings (type, options, use as data point name, etc) are set up properly.
12. Make sure the publish button of the survey is disabled, if not check the messages tab and make sure publish has succeeded.
13. Make a change in one of the questions and publish.
14. Check that the survey was published and version number changed.

Mandatory questions

1. Select another of the created surveys and name it <release version> mandatory questions survey
2. Add a form with one question group
3. Add a free text question with double entry flag checked
4. Add a number question with double entry flag checked
5. Add a picture question
6. Add a geolocation question
7. Ensure all four questions have the mandatory flag checked
8. Add Variable name for all questions
9. Publish

Dependent questions

1. Select the last of the created surveys and name it <release version> dependent questions survey
2. Add a form with two question groups
3. In group one add three option questions with three options (1,2,3)
4. Create a dependency in question 2, which depends on question 1 option 3
5. Create a dependency in question 3 which depends on question 2 option 2
6. In the second question group add two more option questions with the same three options
7. Create a dependency in question 1 which depends on question 2 option 2 from group one
8. Create a dependency in question 2 which depends on question 3 option 3 from group one
9. Publish

Copying

1. In the folder <release version> Non MF Surveys, click to copy the <release version> All questions survey and place the copy in the same folder
2. Repeat this, but place the copy in a different folder
3. Visit the messages tab to see when this is complete
4. When complete, open the copies and confirm everything present - including translations
5. In one of the copies, copy the entire question group and confirm everything looks as expected

Deleting

1. Take the first of the two copies just created
2. Click to delete each question individually
3. Confirm you can delete the form
4. Confirm you can delete the survey when the form is deleted
5. Repeat this with the second copy, except delete all the questions in one go by deleting the whole question group

Devices Tab

Manage Devices

1. Edit the name of an existing group
2. Delete an existing group
3. Create a new group
4. Assign your test device to this new group

Manage assignments

1. Create new assignment for each of the 4 created surveys and add all the forms of that survey.
2. Set the date to the period of testing
3. Add your test device from the previously created group
4. Save assignment

Manual transfer tab

1. Select the three non MF surveys and add their respective forms one by one
2. Enter your email address and click send
3. Confirm bootstrap email received and file can be downloaded

• Known issue: #2804 Manual survey transfer pre-selected forms

App

App betas

In order to get the latest app beta build, open this link https://uat1.akvoflow.org/app2 on your Android device.

Basic Verification

1. If old version of app installed, confirm prompt notifies of new version and that it can be installed
2. If not already installed, confirm the latest version of the apk from /app2 link can be installed
3. Confirm device identifier and instance name set up in preferences correctly (they are located at the bottom). Instance name is the dashboard url, for example https://uat1.akvoflow.org

Survey acquisition

1. If valid assignment created previously, then opening app should fetch all test surveys
2. Click options and select to download form by id
3. Enter the id for a random form - select any from the dashboard
4. Confirm this is fetched

It can be useful to have some sort of fake GPS application installed when performing the following steps, as this prevents all map points appearing in the one place

Monitored surveys

1. Select the MF survey which was assigned
2. Create new data point
3. Complete the registration forms and submit
4. Make sure the data point name is only generated from the free text question response and not also from number question
5. Re enter the data point and complete the other form
6. Submit at least two responses for this form
7. Create another 2+ data points and repeat process
8. Check that 'Search' and 'Sort' work properly when looking at the list of data points.
9. Open the 'Map' tab, check the data points show on the map and click on one of the points, check its 'History'.

Non Monitored Surveys

all questions survey

1. Select the all questions survey
2. Create a new data point and answer all questions
3. Submit
4. Create and submit several more responses, making sure that a few of them include repeated groups
5. Ensure that question types like cascade and signature can not be submitted half completed
6. Check that translations can be selected and are properly shown on the screen (multiple langugages at once, oswitch languages)

dependent questions survey

1. Select the dependent questions survey
2. Create new data point
3. Ensure that the dependencies work as expected
4. Submit

mandatory questions survey

1. Select the mandatory questions survey
2. Create new data point
3. Answer questions, ensuring double entry complains if not matching
4. Submit
5. Repeat for mandatory questions 5+ times (at least 5 data points at end)

Form version check

1. Go to dashboard, refresh the survey tab and open <release version> mandatory questions survey
2. Add two geoshape questions
3. Publish and check the version number is updated
4. Close the app and make sure it is not running in the background of the phone
5. Re-open the app and confirm a form is being automatically downloaded
6. Open <release version> mandatory questions survey and create a new point.
7. Confirm the version of the form is updated in the header and the form holds the 2 new geoshape questions.
8. Submit a few new data points

Bootstrap

1. On your test device within the flow app go to Settings and publish data. Navigate to the SD Card file storage Android/data/org.akvo.flow/files, select the published folder, and then zip it
2. Transfer this to your computer by connecting via usb cable or emailing the zip file, you will need it for Bulk Upload
3. Remove application and delete the akvoflowfolder from your sd card
4. Re-install the application but don't open it
5. Move a bootstrap file to the inbox folder on your sd card
6. Put the phone in airplane mode
7. Open application and confirm surveys are bootstrapped as expected

Data Tab

Charts

1. Select one of the recently completed surveys which contains an option question
2. Check that the various chart types display as expected

Exports (all export types except data cleaning)

Here we will test the Data analysis export, the Comprehensive report, the geoshape data and the survey form.

1. Data / Exports / New export
2. Select the mandatory questions survey you created earlier
3. Click generate on Comprehensive report to export a raw data report (you should receive an email)
4. Export another report but make use of the date filtering for responses, the Most recent submissions radio button should only be clickable when a monitoring form is selected for export.
5. Select the all questions survey you created earlier
6. Check all data is rendered as expected
7. Change option question answers for one submission (for both with codes and without codes) via the Inspect data tab in the dashboard and re-download the report. Check that the answers are rendered properly and reflect the change made.
8. Check that the questions in the report are the same as those in the survey.
9. Add a question to the form but do not publish it.
10. Export the report again. There should not be a column for the added question.
11. Click to export a geographic shape (you'll have to select the shape question from the drop down) - confirm everything looks as expected
12. Click to export a data analysis export - confirm any repeat groups look as expected
13. Click to export a survey form - confirm everything looks as expected

Data Cleaning

The data cleaning can be done either via the dashboard Inspect data tab or exporting and importing a spreadsheet. To test the data cleaning using the spreadsheet, the steps are:

* Data -> Exports -> New export -> select the folder/survey/form -> Data cleaning export
* Once the report is ready download the spreadsheet
* Make the edits and save the file
* Data -> Data cleaning tab -> select the folder/survey/form -> select the spreadsheet to upload -> import

Mandatory questions survey

1. Create and download the Data cleaning export for the mandatory questions survey.
2. Edit all the values from the free text question
3. Move the free text column one to the right (but do not leave an empty column)
4. Reorder the response rows
5. Save and Upload
6. Once the Messages tab reports the spreadsheet has been processed, re-download the export and confirm responses were updated

All questions survey (repeated groups)

1. Create and download the Data cleaning export for the all questions survey.
2. Add another row in between a repeated group for another submission. Add the answers to the row (as if the submission was made on the phone) for the repeated questions. Add the correct repetition number.
3. Repeated the above mentioned step but add a wrong repetition number.
4. Save sheet and import back.
5. Check the messages tab to see if the report has been processed. It should have failed due to missing repeat number.
6. Open the saved spread sheet, remove the fully empty row, save and upload it again.
7. Confirm with the messages tab that the spreadsheet was processed.
8. Download the report via the Export report option and cross check that the changes are visible and all ok.

MF survey (Registration form)

1. Create and download the Data cleaning export for the MF survey > Registration form.

2. Edit the free text question response for a data point by adding in 'NAME CHANGE'

3. Add in 3 new data points - add in a response for the free text question 'NEW - name of city/location on the map' and the location info (latitude, longitude, elevation = 1, geocode = leave empty) from the chosen city/location, add in '1' in the repeated column

4. Save and upload again.

5. Once the message tab reports the spreadsheet has been processed, go to Inspect data tab and confirm the new points are added and the change to the existing point is made.

6. Go to Map tab and confirm the new points are added. Go to the app, selected the MF survey and download the data points. The 3 new points should appear in the list and the existing one should reflect the name change.

   • Known issues and bugs:
     • #2643 After importing in new data, they do not show on the map

Dependent questions survey

1. Go to Inspect data tab and open the dependent questions survey
2. Edit all submissions so in question one the answer is everywhere '1'
3. Create and download the data cleaning export for the dependent questions survey
4. Check that the edits are rendered properly
5. In the spread sheet, change all answers in column for question two into '1' as well and save file
6. Upload the file, check that the Messages tab confirms the upload
7. Go to Inspect data tab and open the dependent questions survey and check that both question one and question two have everywhere answer 1 selected

Inspect data Tab

1. On the Inspect data tab select the non MF surveys
2. Ensure all three surveys have data that appears
3. Chose to edit some of the responses
4. Make sure to try and edit cascade questions
5. Ensure edits save
6. Delete all data for these surveys

Monitoring Tab

1. Select the Monitoring tab, choose the MF survey you created
2. Ensure responses appear and detailed responses can be viewed
3. Delete everything
   • Known issue: https://github.com/akvo/akvo-flow/issues/3021

Bulk Upload

1. Go to Data / Bulk upload data on the dashboard, select this zip file you got from published data on the device and upload
2. Check the messages tab to see when it completes
3. Once complete the previously deleted responses should be restored

• Known issues and bugs:
  • #155 Bulk upload may not work if zip file path is absolute (generated by a mobile app for example)

Resources Tab

Cascade resources

1. Add three new cascades via the import process - making sure that different separator formats are covered (comma, tab, semi-colon) and that codes are present in one file but not the other
2. Check cascades look as expected
3. Delete nodes at various levels, check structure remains as expected
4. Publish one of the unaltered resources
5. Check messages tab to see if publish successful

Maps Tab

1. Check points submitted from latest round of testing are present
2. Check question overlay loads as expected and responses, image questions etc are fine

Users Tab

1. You need two browsers or different sessions to test this
2. The account you have been using so far should be a super admin
3. You need an additional test Google account to restrict permissions on

Super Admin Tests

1. User can see and do everything
2. If second test account is already added to dashboard, remove it and then re-add with the Administrator - Create, Modify and Delete Users checkbox ticked
3. Confirm account appears in user list, edit name etc should all work as expected
4. Go to the Roles and Permissions tab
5. Add a new Administrator role with a couple of permissions ticked
6. Save and verify that the role is created with the appropriate list of permissions
7. Edit the role and tick all the permissions
8. Save and verify that the list is updated to reflect all permissions correctly
9. Create a second role Administrator2 with all permissions ticked
10. Delete the role Administrator2
11. Verify that the role is correctly deleted from the list
12. Return to the users tab and select your test user
13. Assign the Administrator role to this user, against one of the folders you created for this round of testing

Test User Admin Tests

1. Log in as your test user
2. You should only see the two folders you gave permissions for
3. You should be able to perform all actions in these two folders
4. Drop down lists on the data tabs etc should only have the two folders
5. You should be able to see the Users tab

Test User Read Only Tests

1. Log back in as your regular super admin account
2. Remove the Administrator role from the test user account
3. Create a new role ReadOnly which only has the view permissions enabled
4. Assign this to the Test User on the same folders as before
5. Log in as Test User
6. Confirm folders and surveys visible but that it's not possible to do anything other than view
7. Confirm data cleaning etc is not possible, on inspect data you should be able to view but not delete responses

Translations

Select from the various translations available at the bottom right of the dashboard and ensure that all elements on all tabs appear to be correctly translated

Release Issue List

Select the milestone for the release on github (which should contain all the issues which are part of the release candidate). Each issue should have its own test plan, run through this test plan and if everything passes, mark the issue with the "Ready for release" label.