Reproducibility of e3sm_diags runs from zppy

[chengzhuzhang]

In the Output Metadata for each diagnostics plot, it includes the command line that produce the 3 panel plots. When working with @BunnyVon , we found that to run e3sm_diags with zppy, the path parameters only show strings, instead of full path of the data. This makes reproducing figures less straightforward, one has to locate the data path before re-run. Example command shows below:

e3sm_diags lat_lon_land --no_viewer --reference_data_path 'climo_ref_land' --test_data_path 'climo_test_land' --results_dir 'model_vs_model_1985-2014' --case_id 'model_vs_model' --run_type 'model_vs_model' --sets 'lat_lon_land' --variables 'TLAI' --seasons 'DJF' --regions 'global' --regrid_tool 'esmf' --regrid_method 'conservative' --multiprocessing --num_workers '24' --main_title 'TLAI DJF global' --backend 'mpl' --output_format 'png' --canvas_size_w '1212' --canvas_size_h '1628' --figsize '8.5' '11.0' --dpi '150' --arrows --contour_levels '0' '0.412' '0.823' '1.23' '1.65' '2.06' '2.47' '2.88' '3.29' '3.7' '4.12' '4.53' '4.94' '5.35' '5.76' '6.17' --test_name 'v2.LR.BDRDCO2' --short_test_name 'CO2v2AMIP.BDRD' --test_colormap 'WhiteBlueGreenYellowRed.rgb' --ref_name 'v2.LR.BGC-LNDATM.CONTRL' --reference_colormap 'WhiteBlueGreenYellowRed.rgb' --diff_title 'Difference' --diff_colormap 'BrBG' --granulate 'variables' 'seasons' 'plevs' 'regions' --selectors 'sets' 'seasons'

Ideally, we want e3sm_diags to expose the complete path, or we can implement through zppy.

[chengzhuzhang]

I had a brief investigation. There is not a simple solution, because zppy is using symlinks from temporal directories which was later removed after the zppy run. zppy saves climo files from all year ranges in one folder, which e3sm_diags won't know which set of climos to use. An additional step is needed to create symlinks which isolates each set of climo files.

The current workaround, we have is to create the test and reference data directory following steps provided in e3sm_diags scripts from zppy output, e.x.: e3sm_diags_lnd_monthly_mvm_lnd_model_vs_model_1985-2014_vs_1985-2014.o584836. as following example:

mkdir -p climo_test_land
cd climo_test_land
cp -s /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_01_198501_201401_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_02_198502_201402_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_03_198503_201403_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_04_198504_201404_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_05_198505_201405_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_06_198506_201406_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_07_198507_201407_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_08_198508_201408_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_09_198509_201409_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_10_198510_201410_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_11_198511_201411_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_12_198512_201412_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_ANN_198501_201412_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_DJF_198501_201412_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_JJA_198506_201408_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_MAM_198503_201405_climo.nc /lcrc/group/e3sm2/ac.sfeng1/E3SM_Simulations/20240801.v2.LR.F20TRCO2.ne30pg2_r05_EC30to60E2r2.chrysalis/zppy.5yrs/post/lnd/native/clim/30yr/v2.LR.F20TRCO2_SON_198509_201411_climo.nc .

Supplying the test_data_path and ref_data_path in above e3sm_diags command line, and it can be used for reproducing or modifying the three-panel figure for publication.