Add M21C surface met forcing

[gmao-rreichle]

Adds functionality to use M21C surface met forcing.

Related PRs: GEOS-ESM/GEOSldas#794

Met_tag options for M21C forcing:

M21CINT - M21C forcing on 1152-by-721 lat/lon grid (model precip)
M21CCSINT - as in M21CINT but on c360 grid
M21CCOR - M21C forcing on 1152-by-721 lat/lon grid with corrected precip seen by land in M21C
M21CCSCOR - as in M21CCOR but on c360 grid
M21CCOR__PREC[xxx] - as in M21CCOR but with external corrected precip
M21CCSCOR__PREC[xxx] - as in M21CCOR__PREC[xxx] but on c360 grid

For testing, M21C data can be found here:
/home/dao_ops/e5303_m21c_jan98/run/.../archive/diag/
/home/dao_ops/e5303_m21c_jan08/run/.../archive/diag/
/home/dao_ops/e5303_m21c_jan18/run/.../archive/diag/

Testing:
Successfully passed all "nightly" tests after 5d40715
/discover/nobackup/rreichle/SystemTests/logs/LDAS/2025-01-23/log.13:25:25
(Re-testing may be needed if further commits are added.)

[gmao-qliu]

Test cases M21CINT and M21CCOR have passed. The cubedsphere forcing tests with MET_HINTERP:0 failed due to interpolation error. Got the following error messages after reading the first forcing file:
"
opening file: ../input/met_forcing/archive//e5303_m21c_jan98/diag/Y1997/M12/e5303_m21c_jan98.lfo_tavg_1hr_glo_C360x360x6_slv.1997-12-31T2330Z.nc4
get_GEOS 3910
LDAS ERROR (3000) from repair_forcing: Tair too low
get_GEOS 3910
repair_forcing: Rainf < 0. in tile ID 1: met_force(i)%Rainf = -0.99990E+04
repair_forcing: Rainf_C < 0. in tile ID 1: met_force(i)%Rainf_C = -0.99990E+04
repair_forcing: Snowf < 0. in tile ID 1: met_force(i)%Snowf = -0.99990E+04
repair_forcing: Tair < 190.0 in tile ID 1: met_force(i)%Tair = -0.99990E+04
LDAS ERROR (3000) from repair_forcing: Tair too low
"
Note all values are missing. The data in the input file are valid.

[gmao-rreichle]

Note all values are missing. The data in the input file are valid.

Thanks, @gmao-qliu, for the update. If the values in the M21C input file are valid, was there possibly a change in how the coordinates in the nc4 files are handled between GEOS-IT and M21C? But I don't see a difference in the files in this regard; it looks like the netcdf "dimensions" in the GEOS-IT and M21C cube-sphere nc4 files are specified consistently (except c180 vs c360, as expected). Not sure how we end up with the interpolation problem and the no-data-values. @weiyuan-jiang, when you get a chance, can you please take a look? Thanks!

[weiyuan-jiang]

I deleted above comment. It is inaccurate.

[gmao-rreichle]

The problem here is that the cubed-sphere forcing file is using different convention now. It is not n_lon X 6 n_lon , but n X n X 6 with nf=6. We need to adjust how to read the forcing with that format

Thanks, @weiyuan-jiang. I'm still a bit confused because I don't see any difference in this regard between GEOS-IT and M21C output, although I might well be missing something. See image below.

Is the (now outdated) "n_lon X 6 n_lon" format from even older cube-sphere output? When @saraqzhang runs the coupled ADAS/LDAS system, the LDAS reads cube-sphere forcing generated by the ADAS within the cycling system. How does this impact what Sara is doing?

[weiyuan-jiang]

@saraqzhang Hi, Sara, Do you have any forcing data file that works? I would like to compare the data format.

[gmao-rreichle]

I deleted above comment. It is inaccurate.

I had just posted my comment when I got this message. I'm leaving my comment in place because it mentions the cube-sphere reader in the ADAS/LDAS coupled system, which needs to be considered in whatever we do here. Note that we don't have a nightly test for the ADAS/LDAS coupled system, so we wouldn't easily know if we broke anything

[saraqzhang]

@weiyuan-jiang @gmao-rreichle the most recent coupled experiment used the cube-sphere forcing, see a set of files at
/discover/nobackup/qzhang/xhybd_ladas/recycle/holdpredout/*

[weiyuan-jiang]

@weiyuan-jiang @gmao-rreichle the most recent coupled experiment used the cube-sphere forcing, see a set of files at /discover/nobackup/qzhang/xhybd_ladas/recycle/holdpredout/*

I don't have permission to that folder

[weiyuan-jiang]

The problem here is that the cubed-sphere forcing file is using different convention now. It is not n_lon X 6 n_lon , but n X n X 6 with nf=6. We need to adjust how to read the forcing with that format

Thanks, @weiyuan-jiang. I'm still a bit confused because I don't see any difference in this regard between GEOS-IT and M21C output, although I might well be missing something. See image below.

Is the (now outdated) "n_lon X 6 n_lon" format from even older cube-sphere output? When @saraqzhang runs the coupled ADAS/LDAS system, the LDAS reads cube-sphere forcing generated by the ADAS within the cycling system. How does this impact what Sara is doing?

I cannot see the missing or filled data values

[gmao-rreichle]

I cannot see the missing or filled data values

@weiyuan-jiang, not sure to what you're referring with this comment. I didn't mean to show missing or filled values in my comment, simply illustrating the dimensions encoded in the nc4 files from GEOS-IT and M21C. Do you need the locations of the GEOS-IT and/or M21C files?

[gmao-qliu]

@weiyuan-jiang , I did additional experiments. Here are the updates: the job runs fine when the model resolution matches the forcing resolution (i.e., CS360). The errors are the same for both the EASEv2_M36 and CS180 model resolutions. It seems that the spatial interpolation of the forcing is not handled properly with cubed-sphere forcings.

[gmao-rreichle]

the job runs fine when the model resolution matches the forcing resolution (i.e., CS360).

@gmao-qliu: I'm confused. This is the only configuration that is expected to work with cube-sphere forcing. That is, the cube-sphere forcing cannot be used with EASE tile spaces or any tile spaces other than what matches the cube-sphere of the forcing. Sorry I didn't make this clearer in my request for testing. It should be pretty clear in the documentation, but let me know if that needs beefing up

[weiyuan-jiang]

I just finished my debug run. In the err file, it said :

pe=00246 FAIL at line=05702 LDAS_Forcing.F90 <forcing on cube-sphere grid: forcing grid dimension must match native grid dimension (grid associated with tile space)>

So to use the forcing data, the model must run on the same resultions of cs grid.

[gmao-qliu]

@gmao-rreichle Sorry I was unclear of that. I thought it'd work as long as MET_HINTERP is set to 0. Maybe add some clarification this in the exeinp template in the following section.

---- Surface meteorological forcing: Horizonal interpolation

1 : bilinear interpolation (default)

0 : nearest neighbor

When forcing is on CS grid, must set MET_HINTERP: 0

MET_HINTERP: 0

[gmao-rreichle]

Thanks for sorting this out, @gmao-qliu and @weiyuan-jiang.

Besides the use of cube-sphere forcing in the coupled ADAS/LDAS (which is built into the config and documentation of that system), we never had a MET_TAG with cube-sphere forcing for general use. The new M21CCS[xxx] options are the first to offer cube-sphere forcing for general use (never mind that M21C is only being produced and wont' be complete for at least another year or two...).

The requirement to use a matching cube-sphere tile space when using cube-sphere forcing really comes with the choice of MET_TAG, and my preference would be to add the documentation of the requirement with that of MET_TAG (and not with MET_HINTERP). I had already documented the requirement with the new M21CCS[xxx] options in GEOSldas/doc/README.MetForcing_and_BCS.md (e.g., https://github.com/GEOS-ESM/GEOSldas/blob/8161b3ea6f3e180f701679e188208840c7a2e80d/doc/README.MetForcing_and_BCS.md?plain=1#L203)

But I can see how the above documentation is a bit hidden and how the documentation of MET_HINTERP in the "sample exe input" file might be misleading. To clarify a bit more, I added the following commit f930ece, which adds documentation into the "sample exe input" file. Please let me know if this makes sense.

@gmao-qliu : It's good to know the M21C MET_TAG options are all running. Did you also verify that the land states and forcings that are output by GEOSldas in these tests look ok?

cc: @saraqzhang

[gmao-qliu]

@gmao-qliu : It's good to know the M21C MET_TAG options are all running. Did you also verify that the land states and forcings that are output by GEOSldas in these tests look ok?

I verified that the input and output precipitation fields are consistent.