JCB-based obs+bias staging, Jedi class updates, and marine B-matrix refactoring #2992

DavidNew-NOAA · 2024-10-08T14:50:05Z

Description

This PR is a companion to GDASApp PR #1312 (merged) and JCB-GDAS PR #31 (merged).

This PR does three things:

It changes the observation and bias staging for the atmospheric analysis tasks to use JCB templates instead of reading the full JEDI input configuration dictionary in order to construct a list of files to stage. This is cleaner and places fewer constraints on how to initialize the analysis.
The Jedi constructor now takes as input a dictionary that is essentially subset of the task_config dictionary. This makes the code clearer and less opaque and makes debugging easier. Each dictionary is constructed from a YAML file with configuration parameters for each JEDI application that is run.
All JEDI applications and their input YAMLs are now initialized in the initialize job of the AtmAnalysis and AtmEnsAnalysis. Before, in the atmensanl* jobs for example, the LETKF solver was initialized in the atmensanlinitcjob, but the LETKF solver and FV3 increment converter were both initialized and executed in the atmensanlobs and atmensanlfv3inc jobs respectively. This makes more sense in terms of resource allocation.

Addendum:

I'm now rolling in the refactoring of the marine B-matrix task into this PR. That makes it also a companion of GDASApp PR #1346 and JCB-GDAS PR #36.

These new changes introduce the Jedi class and JCB into the marine B-matrix job.

Partially resolves GDASApp issue #1296

Type of change

Bug fix (fixes something broken)
New feature (adds functionality)
Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

Is this a breaking change (a change in existing functionality)? YES
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? NO
- EMC verif-global
- GDAS #1346
- GFS-utils
- GSI
- GSI-monitor
- GSI-utils
- UFS-utils
- UFS-weather-model
- wxflow

How has this been tested?

C96C48_ufs_hybatmDA CI runs successfully
GDASApp jjob tests pass successfully

Checklist

Any dependent changes have been merged and published
My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have documented my code, including function, input, and output descriptions
My changes generate no new warnings
New and existing tests pass with my changes
This change is covered by an existing CI test or a new one has been added
I have made corresponding changes to the system documentation if necessary

…AA-EMC#2862)

…cribing (NOAA-EMC#2862)

…A-EMC#2862)

…e bias correction files from tarball (NOAA-EMC#2862)

…on files using jedi class (NOAA-EMC#2862)

…OAA-EMC#2862)

…ns analysis scripts (NOAA-EMC#2862)

…OAA-EMC#2862)

emcbot · 2024-11-16T06:51:41Z

CI Passed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/2992


Experiment C96_S2SWA_gefs_replay_ics_32534bb9 Completed 1 Cycles: *SUCCESS* at Sat Nov 16 00:52:38 UTC 2024
Experiment C48_ATM_32534bb9 Completed 2 Cycles: *SUCCESS* at Sat Nov 16 02:05:34 UTC 2024
Experiment C48mx500_3DVarAOWCDA_32534bb9 Completed 2 Cycles: *SUCCESS* at Sat Nov 16 02:11:39 UTC 2024
Experiment C48_S2SW_32534bb9 Completed 2 Cycles: *SUCCESS* at Sat Nov 16 03:06:44 UTC 2024
Experiment C48_S2SWA_gefs_32534bb9 Completed 1 Cycles: *SUCCESS* at Sat Nov 16 03:13:49 UTC 2024
Experiment C96C48_hybatmaerosnowDA_32534bb9 Completed 3 Cycles: *SUCCESS* at Sat Nov 16 06:21:06 UTC 2024
Experiment C96_atm3DVar_32534bb9 Completed 3 Cycles: *SUCCESS* at Sat Nov 16 06:27:06 UTC 2024
Experiment C96C48_hybatmDA_32534bb9 Completed 3 Cycles: *SUCCESS* at Sat Nov 16 06:27:08 UTC 2024
Experiment C96C48_ufs_hybatmDA_32534bb9 Completed 3 Cycles: *SUCCESS* at Sat Nov 16 06:51:28 UTC 2024

RussTreadon-NOAA · 2024-11-16T19:07:15Z

Hercules C96C48_hybatmDA failure

Examine log files

/work2/noaa/global/CI/HERCULES/2992/RUNTESTS/COMROOT/C96C48_hybatmDA_32534bb9/logs/2021122106/enkfgdas_fcst_mem001.log
/work2/noaa/global/CI/HERCULES/2992/RUNTESTS/COMROOT/C96C48_hybatmDA_32534bb9/logs/2021122106/enkfgdas_fcst_mem002.log

Both forecast jobs died near the start of the model integration with nan messages

0: PASS: fcstRUN phase 1, n_atmsteps =                9 time is        12.491381
 0:  in atmos_model update, fhzero=   6.00000000000000      fhour=   3.333333
 0:   0.0000000E+00
 0: PASS: fcstRUN phase 2, n_atmsteps =                9 time is         3.004835
 4:
 4: FATAL from PE     4: NaN in input field of mpp_reproducing_sum(_2d), this indicates numerical instability
 4:
 4: Image              PC                Routine            Line        Source
 4: ufs_model.x        00000000056109A9  mpp_mod_mp_mpp_er          72  mpp_util_mpi.inc
 4: ufs_model.x        00000000058DBC7E  mpp_efp_mod_mp_mp         195  mpp_efp.F90
 4: ufs_model.x        0000000005824A7C  mpp_domains_mod_m         143  mpp_global_sum.fh
 4: ufs_model.x        0000000002F8D680  fv_grid_utils_mod        3077  fv_grid_utils.F90
 4: ufs_model.x        0000000002FC6949  fv_mapz_mod_mp_la         794  fv_mapz.F90
 4: libiomp5.so        0000150A777F6F13  __kmp_invoke_micr     Unknown  Unknown
 4: libiomp5.so        0000150A77766664  __kmp_fork_call       Unknown  Unknown
 4: libiomp5.so        0000150A7771FD23  __kmpc_fork_call      Unknown  Unknown
 4: ufs_model.x        0000000002FC1E3B  fv_mapz_mod_mp_la         683  fv_mapz.F90
 4: ufs_model.x        0000000002F69276  fv_dynamics_mod_m         771  fv_dynamics.F90
 4: ufs_model.x        0000000002EB982B  atmosphere_mod_mp         688  atmosphere.F90
 4: ufs_model.x        0000000002DA8B58  atmos_model_mod_m         879  atmos_model.F90
 4: ufs_model.x        0000000002C1F7E7  module_fcst_grid_        1335  module_fcst_grid_comp.F90
 4: ufs_model.x        0000000000ADA2E4  Unknown               Unknown  Unknown
 4: ufs_model.x        0000000000ADE25F  Unknown               Unknown  Unknown
 4: ufs_model.x        000000000114E75A  Unknown               Unknown  Unknown
 4: ufs_model.x        00000000012E3358  Unknown               Unknown  Unknown
 4: ufs_model.x        0000000000ADB73A  Unknown               Unknown  Unknown
 4: ufs_model.x        00000000009ED880  Unknown               Unknown  Unknown
 4: ufs_model.x        0000000000DC5641  Unknown               Unknown  Unknown
 4: ufs_model.x        0000000002C0D8D1  fv3atm_cap_mod_mp        1085  fv3_cap.F90
 4: ufs_model.x        0000000002C0D33B  fv3atm_cap_mod_mp        1034  fv3_cap.F90

This is an odd failure. C96C48_hybatmDA is GSI-based atmospheric DA. This PR does not alter either the forecast model or GSI-based DA.

Install DavidNew-NOAA:feature/jcb-obsbias at 32534bb on Hercules in /work/noaa/da/rtreadon/git/global-workflow/pr2992_final. Set up and run C96C48_hybatmDA CI. C96C48_hybatmDA successfully ran all jobs.

(gdasapp) hercules-login-2:/work/noaa/stmp/rtreadon/EXPDIR/prgsi_pr2992_final$ rocotostat -d prgsi_pr2992_final.db -w prgsi_pr2992_final.xml -c all -s
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201800        Done    Nov 16 2024 11:58:06    Nov 16 2024 13:00:04
202112210000        Done    Nov 16 2024 11:58:06    Nov 16 2024 19:04:02
202112210600        Done    Nov 16 2024 11:58:06    Nov 16 2024 18:44:03

Examine output from /work2/noaa/global/CI/HERCULES/2992/RUNTESTS/COMROOT/C96C48_hybatmDA_32534bb9/. The enkf.x analysis increments for 20211221 06Z are nonphysical

 0:  time level            1
 0:  --------------
 0: ens. mean anal. increment min/max  u   -12204.3056641        7737.84765625
 0: ens. mean anal. increment min/max  v   -11370.2402344        8705.36914062
 0: ens. mean anal. increment min/max  tv   -2873.78125000        4715.19384766
 0: ens. mean anal. increment min/max  q  -0.978207349777       0.497514724731
 0: ens. mean anal. increment min/max  oz  -0.121405441314E-02   0.924816820771E-03
 0: ens. mean anal. increment min/max  ps   -1131.41528320        623.390930176
 0:  time level            2
 0:  --------------
 0: ens. mean anal. increment min/max  u   -10770.1357422        8994.37695312
 0: ens. mean anal. increment min/max  v   -9961.04296875        12255.5898438
 0: ens. mean anal. increment min/max  tv   -2885.21777344        6381.37304688
 0: ens. mean anal. increment min/max  q   -1.11263489723       0.543511331081
 0: ens. mean anal. increment min/max  oz  -0.126553326845E-02   0.859888270497E-03
 0: ens. mean anal. increment min/max  ps   -947.946472168        614.040649414
 0:  time level            3
 0:  --------------
 0: ens. mean anal. increment min/max  u   -9989.00585938        8761.36914062
 0: ens. mean anal. increment min/max  v   -7961.80371094        11510.4707031
 0: ens. mean anal. increment min/max  tv   -3515.27587891        5955.34912109
 0: ens. mean anal. increment min/max  q   -1.03635215759       0.862098753452
 0: ens. mean anal. increment min/max  oz  -0.934842391871E-03   0.881655723788E-03
 0: ens. mean anal. increment min/max  ps   -765.077331543        606.946655273

Compare this with the successful rerun

0:  time level            1
 0:  --------------
 0: ens. mean anal. increment min/max  u   -53.3018913269        49.1635665894
 0: ens. mean anal. increment min/max  v   -52.7409667969        49.9739608765
 0: ens. mean anal. increment min/max  tv   -16.9043731689        9.42595672607
 0: ens. mean anal. increment min/max  q  -0.238635810092E-02   0.335678178817E-02
 0: ens. mean anal. increment min/max  oz  -0.139211806527E-05   0.174592651092E-05
 0: ens. mean anal. increment min/max  ps   -2.33502197266        4.34594726562
 0:  time level            2
 0:  --------------
 0: ens. mean anal. increment min/max  u   -34.5535125732        29.5668640137
 0: ens. mean anal. increment min/max  v   -38.0788955688        39.3475875854
 0: ens. mean anal. increment min/max  tv   -14.1956558228        13.5581207275
 0: ens. mean anal. increment min/max  q  -0.306937331334E-02   0.292858737521E-02
 0: ens. mean anal. increment min/max  oz  -0.134557058118E-05   0.189299998965E-05
 0: ens. mean anal. increment min/max  ps   -2.89700317383        5.60110473633
 0:  time level            3
 0:  --------------
 0: ens. mean anal. increment min/max  u   -33.2891349792        65.5625305176
 0: ens. mean anal. increment min/max  v   -40.9432754517        40.9033317566
 0: ens. mean anal. increment min/max  tv   -18.5473480225        11.8623199463
 0: ens. mean anal. increment min/max  q  -0.335096800700E-02   0.276062171906E-02
 0: ens. mean anal. increment min/max  oz  -0.137578490467E-05   0.206520826396E-05
 0: ens. mean anal. increment min/max  ps   -3.58447265625        7.21514892578

Comparison of the enkfstat files from the two runs show no differences apart from timing statistics, different order of printout, and ensemble mean analysis increments.

I have not seen this error in my many runs of C96C48_hybatmDA on Hercules. A rerun of C96C48_hybatmDA via automated CI is likely to succeed. This, however, sidesteps the above failure. Why / how nonphysical increments were generated in this case remains unresolved.

RussTreadon-NOAA · 2024-11-18T11:50:15Z

@WalterKolczynski-NOAA - What's the path forward from here? We would like to get this PR merged into develop sooner rather than later.

I do not see a way to only run C96C48_hybatmDA via git CI labels. It's a waste of time and resources to rerun all configurations on Hercules. Do we accept the explanation above, manually rerun C96C48_hybatmDA, or modify g-w CI to only run a single configuration?
WCOSS2 dev (Dogwood) goes down tomorrow (12/19). Do we launch g-w CI on Dogwood today? My understanding is that WCOSS2 g-w CI is manual, not automatic. Is this correct?
Do we need to run CI on Orion?

RussTreadon-NOAA · 2024-11-18T15:27:13Z

@aerorahul, @WalterKolczynski-NOAA , and @KateFriedman-NOAA

I manually launched g-w DA CI on WCOSS2 (Dogwood). We know C96C48_hybatmaerosnowDA and C48mx500_3DVarAOWCDA will fail. GDASApp must be built with spack-stack on WCOSS2 in order for select jobs in these configurations to successfully run to complete. GSI and JEDI based atmospheric DA, C96C48_hybatmDA and C96C48_ufs_hybatmDA, should pass.

The following g-w CI configuration do NOT contain a wcoss2 skip

C48_ATM.yaml
C48_S2SW.yaml
C96_atm3DVar_extended.yaml

Do these three configurations also need to be run on Dogwood?

WalterKolczynski-NOAA · 2024-11-19T03:33:40Z

@WalterKolczynski-NOAA - What's the path forward from here? We would like to get this PR merged into develop sooner rather than later.

I do not see a way to only run C96C48_hybatmDA via git CI labels. It's a waste of time and resources to rerun all configurations on Hercules. Do we accept the explanation above, manually rerun C96C48_hybatmDA, or modify g-w CI to only run a single configuration?

WCOSS2 dev (Dogwood) goes down tomorrow (12/19). Do we launch g-w CI on Dogwood today? My understanding is that WCOSS2 g-w CI is manual, not automatic. Is this correct?

Do we need to run CI on Orion?

Unfortunately, the CI does not (yet) have the capability to select individual tests. We have to rerun them all on a platform.

Orion is not needed. Orion's support level has been downgraded and most of the things that would cause a failure on Orion will also cause a failure on Hercules anyway.

@aerorahul, @WalterKolczynski-NOAA , and @KateFriedman-NOAA

I manually launched g-w DA CI on WCOSS2 (Dogwood). We know C96C48_hybatmaerosnowDA and C48mx500_3DVarAOWCDA will fail. GDASApp must be built with spack-stack on WCOSS2 in order for select jobs in these configurations to successfully run to complete. GSI and JEDI based atmospheric DA, C96C48_hybatmDA and C96C48_ufs_hybatmDA, should pass.

The following g-w CI configuration do NOT contain a wcoss2 skip

C48_ATM.yaml

C48_S2SW.yaml

C96_atm3DVar_extended.yaml

Do these three configurations also need to be run on Dogwood?

Any tests that aren't being skipped should be run. It looks like the upgrade it postponed, so we should be able to get these in. I will run them overnight so hopefully they are complete by morning.

WalterKolczynski-NOAA · 2024-11-19T04:30:43Z

CI Tests set up to run in /lfs/h2/emc/ptmp/walter.kolczynski/PR/PR_2992/RUNTESTS on WCOSS

emcbot · 2024-11-19T10:04:55Z

CI Passed on Hercules in Build# 3
Built and ran in directory /work2/noaa/global/CI/HERCULES/2992


Experiment C48_ATM_32534bb9 Completed 2 Cycles: *SUCCESS* at Tue Nov 19 01:50:40 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_32534bb9 Completed 1 Cycles: *SUCCESS* at Tue Nov 19 02:08:56 CST 2024
Experiment C96_atm3DVar_32534bb9 Completed 3 Cycles: *SUCCESS* at Tue Nov 19 03:21:28 CST 2024
Experiment C96C48_hybatmDA_32534bb9 Completed 3 Cycles: *SUCCESS* at Tue Nov 19 03:21:33 CST 2024
Experiment C48_S2SW_32534bb9 Completed 2 Cycles: *SUCCESS* at Tue Nov 19 03:39:36 CST 2024
Experiment C48_S2SWA_gefs_32534bb9 Completed 1 Cycles: *SUCCESS* at Tue Nov 19 04:04:44 CST 2024

RussTreadon-NOAA · 2024-11-19T11:03:38Z

WCOSS2 g-w CI
Install DavidNew-NOAA:feature/jcb-obsbias at 32534bb on Dogwood. Run the following g-w CI configurations

C96C48_hybatmDA - prgsi_pr2992
C96C48_ufs_hybatmDA - prjedi_pr2992
C96C48_hybatmaerosnowDA - praero_pr2992
C48mx500_3DVarAOWCDA - prwcda_pr2992
C48_ATM - pratm_pr2992
C48_S2SW - prswsw_pr2992
C96_atm3DVar_extended - pratm3dvar_pr2992

with the following results

/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/prgsi_pr2992
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Nov 18 2024 15:05:08    Nov 18 2024 15:25:14
202112210000        Done    Nov 18 2024 15:05:08    Nov 18 2024 20:55:05
202112210600        Done    Nov 18 2024 15:05:08    Nov 18 2024 20:35:11
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/prjedi_pr2992
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202402231800        Done    Nov 18 2024 15:05:19    Nov 18 2024 15:25:35
202402240000        Done    Nov 18 2024 15:05:19    Nov 18 2024 21:05:15
202402240600        Done    Nov 18 2024 15:05:19    Nov 18 2024 21:05:15
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/praero_pr2992
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200      Active    Nov 18 2024 15:05:30             -          
202112201800      Active    Nov 18 2024 15:05:30             -          
202112210000      Active    Nov 18 2024 15:05:30             -          
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/prwcda_pr2992
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241200        Done    Nov 18 2024 15:05:36    Nov 18 2024 15:25:48
202103241800      Active    Nov 18 2024 15:05:36             -          
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/pratm_pr2992
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Nov 18 2024 16:35:30    Nov 18 2024 20:50:15
202103231800        Done    Nov 18 2024 16:35:30    Nov 18 2024 21:00:16
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/pratm3dvar_pr2992
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Nov 18 2024 16:35:33    Nov 18 2024 17:20:21
202112210000        Done    Nov 18 2024 16:35:33    Nov 19 2024 00:00:22
202112210600        Done    Nov 18 2024 16:35:33    Nov 19 2024 01:40:22
202112211200        Done    Nov 18 2024 17:25:29    Nov 19 2024 04:05:16
202112211800        Done    Nov 19 2024 00:05:15    Nov 19 2024 07:35:15
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/prs2sw_pr2992
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Nov 18 2024 16:35:35    Nov 18 2024 20:50:26
202103231800        Done    Nov 18 2024 16:35:35    Nov 18 2024 21:00:29

All jobs in all configurations except C96C48_hybatmaerosnowDA and C48mx500_3DVarAOWCDA successfully ran to completion.

Note that C96C48_hybatmaerosnowDA and C48mx500_3DVarAOWCDA are set to be skipped on WCOSS2. If GDASApp is rebuilt and run with spack-stack/1.6.0 these two configurations successfully run all jobs to completion.

The following configurations in ci/cases/pr list wcoss2 under the skip_ci_on_hosts: section

./C96C48_hybatmaerosnowDA.yaml:  - wcoss2
./C96C48_ufs_hybatmDA.yaml:  - wcoss2
./C96_S2SWA_gefs_replay_ics.yaml:  - wcoss2
./C48mx500_3DVarAOWCDA.yaml:  - wcoss2
./C48mx500_hybAOWCDA.yaml:  - wcoss2
./C96_atm3DVar.yaml:  - wcoss2
./C48_S2SWA_gefs.yaml:  - wcoss2

These ci configurations, apart from the three configurations which include JEDI based DA, were not run on Dogwood.

Based on the above g-w CI results we can change the CI-Wcoss2-Running label to CI-Wcoss2-Passed.

RussTreadon-NOAA · 2024-11-19T11:05:56Z

Acceptable (explainable) g-w CI results have been obtained on Hera, Hercules, and WCOSS2 (Dogwood).

@WalterKolczynski-NOAA , @aerorahul , @KateFriedman-NOAA : Are any more tests required before this PR can be merged into develop?

RussTreadon-NOAA · 2024-11-19T11:21:26Z

Thank you @WalterKolczynski-NOAA

…efactoring (NOAA-EMC#2992) This PR is a companion to GDASApp PR NOAA-EMC/GDASApp#1312 (merged) and NOAA-EMC/jcb-gdas#31 (merged). This PR does three things: 1. It changes the observation and bias staging for the atmospheric analysis tasks to use JCB templates instead of reading the full JEDI input configuration dictionary in order to construct a list of files to stage. This is cleaner and places fewer constraints on how to initialize the analysis. 2. The ```Jedi``` constructor now takes as input a dictionary that is essentially subset of the ```task_config``` dictionary. This makes the code clearer and less opaque and makes debugging easier. Each dictionary is constructed from a YAML file with configuration parameters for each JEDI application that is run. 3. All JEDI applications and their input YAMLs are now initialized in the initialize job of the ```AtmAnalysis``` and ```AtmEnsAnalysis```. Before, in the ```atmensanl*``` jobs for example, the LETKF solver was initialized in the ```atmensanlinit```cjob, but the LETKF solver and FV3 increment converter were both initialized and executed in the ```atmensanlobs``` and ```atmensanlfv3inc``` jobs respectively. This makes more sense in terms of resource allocation. Addendum: I'm now rolling in the refactoring of the marine B-matrix task into this PR. That makes it also a companion of NOAA-EMC/GDASApp#1346 and NOAA-EMC/jcb-gdas#36. These new changes introduce the ```Jedi``` class and JCB into the marine B-matrix job. Partially resolvesNOAA-EMC/GDASApp#1296 --------- Co-authored-by: RussTreadon-NOAA <[email protected]> Co-authored-by: Rahul Mahajan <[email protected]>

* develop: Remove WAFS files and references from `develop` (NOAA-EMC#3263) fix intel stack version number on c5 (NOAA-EMC#3258) Update gsi_monitor and ufs_utils hashes to recent hashes for C5/C6 build and run (NOAA-EMC#3252) Enable DA cycling on gaea C5/C6 (NOAA-EMC#3255) Copy post-processed sea ice increment for diagnostics (NOAA-EMC#3235) Only run METplus in the 3Dvar tests (NOAA-EMC#3245) Clone, build, and run C48_ATM and C48_S2SW on Gaea C5 and C6 (NOAA-EMC#3106) Add echgres as a dependency only for RUN=enkfgdas, not enkfgfs (NOAA-EMC#3246) Add domain level to wave gridded COM path (NOAA-EMC#3137) CI JJOB Tests using CMake (NOAA-EMC#3214) Make assorted updates to waves (NOAA-EMC#3190) Move WCOSS2 LD_LIBRARY_PATH patches to load_ufsda_modules.sh (NOAA-EMC#3236) Adding a gefs_arch task to GEFS workflow (NOAA-EMC#3211) Add additional GEFS variables needed for AI/ML applications (NOAA-EMC#3221) Add bmat task dependency to marine LETKF task (NOAA-EMC#3224) Resolve bug with LMOD_TMOD_FIND_FIRST setting affecting build on WCOSS2 (NOAA-EMC#3229) Reinstate product groups (NOAA-EMC#3208) Additional fixes for downstream jobs (NOAA-EMC#3187) Turn IAU off during staging job for cold start experiments (NOAA-EMC#3215) Update the gdas.cd hash and enable GDASApp to run on WCOSS2 (NOAA-EMC#3220) Update upload-artifact to v4 (NOAA-EMC#3216) Prevent duplicate case generation in generate_workflows.sh (NOAA-EMC#3217) Update g-w to cycle with C1152 ATM (NOAA-EMC#3206) Separate use of initial increment/perturbation file from REPLAY/+03 ICs (NOAA-EMC#3119) Update gsi_enkf hash and gsi_ver (NOAA-EMC#3207) Remove cpus-per-task from APRUN_OCNANALECEN on WCOSS2 (NOAA-EMC#3212) Remove 5WAVH from AWIPS GRIB2 parm files (NOAA-EMC#3146) Remove multi-grid wave support (NOAA-EMC#3188) Add echgres as a dependency for earc (NOAA-EMC#3202) Ensure OCNRES and ICERES have 3 digits in the archive script (NOAA-EMC#3199) Set runtime shell requirements within Jenkins Pipeline (NOAA-EMC#3171) Add efcs and epos to ufs_hybatm xml (NOAA-EMC#3192) (NOAA-EMC#3193) Fix GEFS and SFS compile flags in build_all.sh (NOAA-EMC#3197) Remove early-cycle EnKF forecast (NOAA-EMC#3185) Fix mod_icec bug in atmos_prod (NOAA-EMC#3167) Create compute build option (NOAA-EMC#3186) Support global-workflow using Rocky 8 on CSPs (NOAA-EMC#2998) Change orog gravity wave drag scheme for grid sizes less than 10km (NOAA-EMC#3175) Switch snow DA to use 2DVar for deterministic and ensemble mean (NOAA-EMC#3163) Update compression options for GEFS history files (NOAA-EMC#3184) Update compression options for high res history files (NOAA-EMC#3178) Turn DO_TEST_MODE off (NOAA-EMC#3177) Hotfix for gdas_arch div/0 (NOAA-EMC#3169) Allow building of the ufs-weather-model, WW3 pre/post execs for GFS, GEFS, SFS in the same clone of global-workflow (NOAA-EMC#3098) Switch Aerosol DA to use JCB and Jedi class (NOAA-EMC#3125) Update ufs-weather-model to 2024-12-06 commit (NOAA-EMC#3145) Enable traditional threading as an option (NOAA-EMC#3149) Update HPC_ACCOUNT on Hercules to fv3-cpu (NOAA-EMC#3164) Turn C96C48_ufs_hybatmDA and C48mx500_3DVarAOWCDA into a regression test (NOAA-EMC#3120) Update GSI analysis jobs to use COMIN/COMOUT (NOAA-EMC#3092) Update HPC Tier Definitions (NOAA-EMC#3138) Add marine hybrid envar (NOAA-EMC#3041) Archive the experiment directory along with git status/diff output (NOAA-EMC#3105) Use stochastic restart patterns on rerun (NOAA-EMC#3077) Point Jenkinsfile back to CI/ (NOAA-EMC#3139) Fix wave restart for cold start and add ic version file (NOAA-EMC#3112) Allow users to override the default account at setup time (NOAA-EMC#3127) Refactor gridded wave post (NOAA-EMC#3014) Update docs related to NOAA CSPs (NOAA-EMC#3043) Allow APP to differ between RUNs (NOAA-EMC#2943) Run one executable for soca2cice (instead of two) (NOAA-EMC#3118) Speed up GSI analysis jobs in CI testing (NOAA-EMC#3115) Make aerosol output frequency variable (NOAA-EMC#2982) Add new stations to GFS BUFR sounding products (NOAA-EMC#3107) JCB-based obs+bias staging, Jedi class updates, and marine B-matrix refactoring (NOAA-EMC#2992) Enable tapering of atm ens perts at the model top (NOAA-EMC#3097) Update JGDAS ENKF POST job (NOAA-EMC#3090) SFS Runs at C96mx100 (NOAA-EMC#2960) Move machine-based options from config.base to host files (NOAA-EMC#3053) Remove RUNDIRS before running CI cases to cover re-run events (NOAA-EMC#3076) CI GitHub pipeline (hotfix) update for fetching repo name (NOAA-EMC#3084) Update JGDAS ENKF ECEN job (NOAA-EMC#3050) Update snow obs processing job (NOAA-EMC#3055) Update to action workflow pipeline in default repo for development (NOAA-EMC#3062) Update to action workflow pipeline in default repo for development (NOAA-EMC#3061) Update workflow pipeline (NOAA-EMC#3060) PW CI pipeline update5 ready for review so it can be merged and tested (NOAA-EMC#3059) Revert "GitHub CI Pipeline update for debugging forked PR support" (NOAA-EMC#3057) GitHub CI Pipeline update for debugging forked PR support (NOAA-EMC#3056) Add more ocean variables for post-processing in GEFS (NOAA-EMC#2995) Auto provisioning of PW clusters from GitHub CI added (NOAA-EMC#3051) Fix the name of the TC tracker filenames in archive.py (NOAA-EMC#3030) Make wxflow links static instead of from link_workflow (NOAA-EMC#3008) Update global jdas enkf diag job with COMIN/COMOUT for COM prefix (NOAA-EMC#2959) Add run and finalize methods to marine LETKF task (NOAA-EMC#2944) Fix wave restarts and GEFS FHOUT/FHMAX (NOAA-EMC#3009) Disabling hyper-threading (NOAA-EMC#2965) GitHub Actions Pipeline Updates for Self-Hosted Runners on PW (NOAA-EMC#3018) CI jekninsfile update hotfix (NOAA-EMC#3038) Update gdas.cd (NOAA-EMC#2978) Add ability to add tag to pslots with generate_workflows (NOAA-EMC#3036) CI update to shell environment with HOMEgfs to HOME_GFS for systems that need the path (NOAA-EMC#3013) Quick updated to Jenkins (health check) launch script (NOAA-EMC#3033) Document the generate_workflows.sh script (NOAA-EMC#3028) Replace gfs_cyc with an interval (NOAA-EMC#2928) Hotfix: Fix generate_workflows.sh optional build flags (NOAA-EMC#3024) Add a tool to run multiple YAML cases locally (NOAA-EMC#3004) Hotfix: Correctly set overwrite option when specified (NOAA-EMC#3021)

RussTreadon-NOAA and others added 30 commits August 28, 2024 17:01

replace separate JEDI radiance bias correction files with tarball (NO…

20fb88c

…AA-EMC#2862)

Merge branch 'develop' into feature/radbcor

8a219cd

rename JEDI radiance bias correction tarball file to be more self-des…

73aa233

…cribing (NOAA-EMC#2862)

Merge remote-tracking branch 'upstream/develop' into feature/radbcor

00a013f

Merge branch 'NOAA-EMC:develop' into feature/radbcor

3215d9b

Merge branch 'develop' into feature/radbcor

04b4347

update gdas.cd hash to include recent commits to GDASApp develop (NOA…

1203866

…A-EMC#2862)

update pygfs jedi class and atmospheric tasks to extract JEDI radianc…

5985e9f

…e bias correction files from tarball (NOAA-EMC#2862)

Merge branch 'develop' into feature/radbcor

009659c

move radiance bias correction staging to jedi class (NOAA-EMC#2868)

73e56a7

initial attempt to generalize processing of variational bias correcti…

666d7d6

…on files using jedi class (NOAA-EMC#2862)

move VarBcDir inside initialize_analysis method (NOAA-EMC#2862)

1e4a284

Merge branch 'develop' into feature/radbcor

386313b

Merge branch 'develop' into feature/radbcor

dd9099e

update sorc/gdas.cd hash to bring in radiance tarball ctest changes (N…

b48e126

…OAA-EMC#2862)

Merge branch 'develop' into feature/radbcor

27434bb

add extract_tar method to jedi class, use extract_tar in atm and atme…

ba77d4e

…ns analysis scripts (NOAA-EMC#2862)

correct typo in atmens_analysis.py (2862)

bc3a6dd

Merge branch 'develop' into feature/radbcor

7ba33c1

Merge branch 'develop' into feature/radbcor

7182a2a

Merge branch 'develop' into feature/radbcor

8016852

Merge branch 'NOAA-EMC:develop' into feature/radbcor

4fb8f81

Merge branch 'develop' into feature/radbcor

bdffc02

improve error handling for jedi class tarfile.extractall (NOAA-EMC#2862)

0ab7bdd

remove unnecessary close following tarfile add (NOAA-EMC#2862)

8b627a1

use endswith to clean up scripting (NOAA-EMC#2862)

eb4fc37

Merge branch 'develop' into feature/radbcor

4229886

refactor jedi class method extract_tar (NOAA-EMC#2862)

ed52b72

update sorc/gdas.cd to current head of GDASApp develop (NOAA-EMC#2862)

80305b6

make jedi method extract_tar static, clean up comments, correct typo (N…

821ee76

…OAA-EMC#2862)

emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules labels Nov 19, 2024

WalterKolczynski-NOAA added CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Nov 19, 2024

RussTreadon-NOAA added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Nov 19, 2024

WalterKolczynski-NOAA merged commit 7ff942e into NOAA-EMC:develop Nov 19, 2024
10 of 11 checks passed

DavidNew-NOAA deleted the feature/jcb-obsbias branch November 19, 2024 13:51

RussTreadon-NOAA mentioned this pull request Nov 19, 2024

Resume nightly testing of GDASApp NOAA-EMC/GDASApp#1313

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JCB-based obs+bias staging, Jedi class updates, and marine B-matrix refactoring #2992

JCB-based obs+bias staging, Jedi class updates, and marine B-matrix refactoring #2992

DavidNew-NOAA commented Oct 8, 2024 •

edited

Loading

emcbot commented Nov 16, 2024

RussTreadon-NOAA commented Nov 16, 2024

RussTreadon-NOAA commented Nov 18, 2024

RussTreadon-NOAA commented Nov 18, 2024

WalterKolczynski-NOAA commented Nov 19, 2024

WalterKolczynski-NOAA commented Nov 19, 2024

emcbot commented Nov 19, 2024

RussTreadon-NOAA commented Nov 19, 2024

RussTreadon-NOAA commented Nov 19, 2024

RussTreadon-NOAA commented Nov 19, 2024

JCB-based obs+bias staging, Jedi class updates, and marine B-matrix refactoring #2992

JCB-based obs+bias staging, Jedi class updates, and marine B-matrix refactoring #2992

Conversation

DavidNew-NOAA commented Oct 8, 2024 • edited Loading

Description

Type of change

Change characteristics

How has this been tested?

Checklist

emcbot commented Nov 16, 2024

RussTreadon-NOAA commented Nov 16, 2024

RussTreadon-NOAA commented Nov 18, 2024

RussTreadon-NOAA commented Nov 18, 2024

WalterKolczynski-NOAA commented Nov 19, 2024

WalterKolczynski-NOAA commented Nov 19, 2024

emcbot commented Nov 19, 2024

RussTreadon-NOAA commented Nov 19, 2024

RussTreadon-NOAA commented Nov 19, 2024

RussTreadon-NOAA commented Nov 19, 2024

DavidNew-NOAA commented Oct 8, 2024 •

edited

Loading