README.test

CICE version 6 testing
  June 2017

This README file documents how to use the testing suite developed for the
CICE Consortium sea ice model.  The tests currently supported are:
  1. "Smoke" test  - Ensures that the model compiles successfully and runs
                     to completion for a 1 day simulation.  Validation is
                     determined by performing a binary comparison on restart
                     files to a baseline dataset.  This test is created by
                     passing 'smoke' to the '-t' argument.
  2. 10-day test   - Ensures that the model compiles successfully and runs
                     to completion for a 10 day simulation.  Validation is
                     determined by performing a binary comparison on restart
                     files to a baseline dataset.  This test is created by 
                     passing '10day' to the '-t' argument.
  3. Exact restart - Ensures that the output from a cold-start simulation
                     and a restart simulation provide bit-for-bit identical
                     results.  No baseline dataset needs to be created for 
                     this test, as the baseline and test datasets are computed
                     at the same time.  This test is created by passing 'restart'
                     to the '-t' argument.
  4. Annual test   - Ensures that the model compiles successfully and runs
                     to completion for an annual simulation.  Validation is
                     determined by performing a binary comparison on restart
                     files to a baseline dataset.  This test is created by 
                     passing 'annual' to the '-t' argument.

Please run './create.case -h' for additional details.

--- To generate a baseline dataset for a test case ---

Quickstart (example):

./create.case -t smoke -m conrad -b
cd smoke_gx3_conrad_4x1
./cice.build
./cice.submit
# After job finishes, check output
cat test_output


--- To run a test case and compare to a baseline dataset ---

Quickstart (example):

./create.case -t smoke -m conrad
cd smoke_gx3_conrad_4x1.t00
./cice.build
./cice.submit
# After job finishes, check output
cat test_output


--- To run a regression test (compare cicev6.0.1 output to cicev6.0.0 output) ---

Quickstart (example):

./create.case -t smoke -m conrad -b -bg cicev6.0.1 -bc cicev6.0.0
cd smoke_gx3_conrad_4x1
./cice.build
./cice.submit
# After job finishes, check output
cat test_output


Additional Details:
- The available options for '-t' are 'smoke', '10day', 'annual', and 'restart'.
- For a 'restart' test, no baseline dataset needs to be generated or compared against.
- The '-b' flag can be combined with '-bd' to specify the location where you 
    want the baseline dataset to be written.  Without specifying '-bd', the
    baseline dataset will be written to the default baseline directory found
    in the env.<machine> file (CICE_MACHINE_BASELINE).
- If neither '-b' or '-bd' are passed, the scripts will look for baseline datasets
    in the default baseline directory found in the env.<machine> file (CICE_MACHINE_BASELINE).
    If the '-bd' option is passed, the scripts will look for baseline datasets in the
    location passed to the -bd argument.
- To generate a baseline dataset for a specific version (for regression testing),
    use '-b' in conjunction to '-bg <version_name>'.  The scripts will then place
    the baseline dataset in $CICE_MACHINE_BASELINE/<version_name>/
- The '-testid' flag allows users to specify a testing id that will be added to the
    end of the case directory.  For example, "./create.case -m conrad -t smoke -testid t12"
    creates the directory smoke_gx3_conrad_4x1.t12.  The default test id is "t00".