Trouble reading NetCDF forecast file

[jwraby]

I created a NetCDF forecast file (netcdf version 4) using a Python script and I tried to run Point-Stat to verify it against point obs. I am encountering an error when it tries to read the file:
ERROR : Trouble reading forecast file.
I'm not sure where to start to resolve this error and would appreciate any assistance you can provide.
Thanks.
(I tried to upload the NetCDF file in GitHub, but it reported "We don't support that file type". I dumped the contents into a text file and uploaded it here.)
TMPanal_d03.txt

[jprestop]

Hi @jwraby. 

Thanks for sending the file in a text format.  I was able to convert it to NetCDF using the command:

ncgen -o TMPanal_d03.nc TMPanal_d03.txt

MET reads a few different flavors of NetCDF files:

• it's internal format which is output from tools like gen_vx_mask or pcp_combine
• those following the Climate Forecast (CF) convention
• WRF output post-processed by the WRF-interp utility

I ran the NetCDF file through plot_data_plane with this command:

plot_data_plane TMPanal_d03.nc TMPanal_d03.ps 'name="TMPvar"; level="(*,*)";' -v 4

which only told me:

DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_None".
ERROR  :
ERROR  : plot_data_plane -> file "TMPanal_d03.nc" not a valid data file

I know that I can set the file_type (see documentation here: https://met.readthedocs.io/en/develop/Users_Guide/config_options.html?highlight=file_type#settings-common-to-multiple-tools).  Knowing that your NetCDF file was not internally generated from MET or produced by the WRF-interp utility, I tried "file_type = NETCDF_NCCF;" to see if it followed the Climate Forecast (CF) convention. 

When I ran plot_data_plane in this way:

/d1/projects/MET/MET_releases/MET/met/bin/plot_data_plane TMPanal_d03.nc TMPanal_d03.ps 'name="TMPvar"; level="(*,*)"; file_type=NETCDF_NCCF;' -v 4

I received the following information:

DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_NcCF".
DEBUG 4: NcCfFile::open() -> could not extract valid time from the "time" variable.
DEBUG 4: NcCfFile::open() -> could not extract valid time from file name.
WARNING:
WARNING: NcCfFile::open() -> could not determine the valid time, using 0.
WARNING:
DEBUG 4: NcCfFile::open() -> could not extract init time from the "forecast_reference_time" variable.
DEBUG 4: NcCfFile::open() -> could not extract init time from file name.
DEBUG 4: NcCfFile::get_grid_from_dimensions() -> The coordinate variable "lat" does not exist.
DEBUG 4: NcCfFile::get_grid_from_dimensions() -> The coordinate variable "lon" does not exist.
ERROR  :
ERROR  : NcCfFile::read_netcdf_grid() -> Couldn't figure out projection from information in netCDF file.
ERROR  : 

So it looks like there are still some issues that need to be worked out in order for the file to follow the Climate Forecast (CF) convention so that MET can read it.  I am no NetCDF expert, but I do know there is a CF-convention checker that you can access here https://cfconventions.org/compliance-checker.html which will hopefully help get you started.

I hope this helps. 

[jwraby]

CLASSIFICATION: UNCLASSIFIED Hi Julie - Thanks for all your diagnostic work with the file. This information is very helpful for identifying the issues which we will need to address in the file as well as for learning your techniques for probing the file and for extracting the diagnostic info. We now have some work to do here. R/ John From: jprestop ***@***.***> Sent: Tuesday, August 17, 2021 9:44 AM To: dtcenter/METplus ***@***.***> Cc: Raby, John W CIV USARMY DEVCOM ARL (USA) ***@***.***>; Mention ***@***.***> Subject: [Non-DoD Source] Re: [dtcenter/METplus] Trouble reading NetCDF forecast file (#1087) All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.

…

_____ Hi @jwraby < Caution-https://github.com/jwraby > . Thanks for sending the file in a text format. I was able to convert it to NetCDF using the command: ncgen -o TMPanal_d03.nc TMPanal_d03.txt MET reads a few different flavors of NetCDF files: * it's internal format which is output from tools like gen_vx_mask or pcp_combine * those following the Climate Forecast (CF) convention * WRF output post-processed by the WRF-interp utility I ran the NetCDF file through plot_data_plane with this command: plot_data_plane TMPanal_d03.nc TMPanal_d03.ps 'name="TMPvar"; level="(*,*)";' -v 4 which only told me: DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_None". ERROR : ERROR : plot_data_plane -> file "TMPanal_d03.nc" not a valid data file I know that I can set the file_type (see documentation here: Caution-https://met.readthedocs.io/en/develop/Users_Guide/config_options.htm l?highlight=file_type#settings-common-to-multiple-tools). < Caution-https://met.readthedocs.io/en/develop/Users_Guide/config_options.htm l?highlight=file_type#settings-common-to-multiple-tools).%C2%A0 > Knowing that your NetCDF file was not internally generated from MET or produced by the WRF-interp utility, I tried "file_type = NETCDF_NCCF;" to see if it followed the Climate Forecast (CF) convention. When I ran plot_data_plane in this way: /d1/projects/MET/MET_releases/MET/met/bin/plot_data_plane TMPanal_d03.nc TMPanal_d03.ps 'name="TMPvar"; level="(*,*)"; file_type=NETCDF_NCCF;' -v 4 I received the following information: DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_NcCF". DEBUG 4: NcCfFile::open() -> could not extract valid time from the "time" variable. DEBUG 4: NcCfFile::open() -> could not extract valid time from file name. WARNING: WARNING: NcCfFile::open() -> could not determine the valid time, using 0. WARNING: DEBUG 4: NcCfFile::open() -> could not extract init time from the "forecast_reference_time" variable. DEBUG 4: NcCfFile::open() -> could not extract init time from file name. DEBUG 4: NcCfFile::get_grid_from_dimensions() -> The coordinate variable "lat" does not exist. DEBUG 4: NcCfFile::get_grid_from_dimensions() -> The coordinate variable "lon" does not exist. ERROR : ERROR : NcCfFile::read_netcdf_grid() -> Couldn't figure out projection from information in netCDF file. ERROR : So it looks like there are still some issues that need to be worked out in order for the file to follow the Climate Forecast (CF) convention so that MET can read it. I am no NetCDF expert, but I do know there is a CF-convention checker that you can access here Caution-https://cfconventions.org/compliance-checker.html which < Caution-https://cfconventions.org/compliance-checker.html%C2%A0which > will hopefully help get you started. I hope this helps. - You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < Caution-#1087 (comment) nt-1195780 > , or unsubscribe < Caution-https://github.com/notifications/unsubscribe-auth/AUX5FH3ZAJRMVXMAOW VMR3LT5J7TFANCNFSM5CIKNWPQ > . Triage notifications on the go with GitHub Mobile for iOS < Caution-https://apps.apple.com/app/apple-store/id1477376905?ct=notification- email&mt=8&pt=524675 > or Android < Caution-https://play.google.com/store/apps/details?id=com.github.android&utm _campaign=notification-email > . <Caution-https://github.com/notifications/beacon/AUX5FH57N6BOAVY75TZQSC3T5J7 TFA5CNFSM5CIKNWP2YY3PNVWWK3TUL52HS4DFWFCGS43DOVZXG2LPNZBW63LNMVXHJKTDN5WW2ZL OORPWSZGOAAJD6BA.gif> CLASSIFICATION: UNCLASSIFIED