Processing wind through point_stat

[gkhboiler]

Hi,

I am new to using the MET software and am confused on how to create wind statistics. I am comparing WRF output to point statistics. I am trying to get a temperature and wind speed/direction verification. I think from what I read, I need to get a mixed pair set of speeds and direction to get the stat I am supposed to create. The good...I seem to be doing okay on the WRF side and the scalar temerature side. When running point_stat I used
fcst = {
file_type = NETCDF_PINT;
field = [
{
name = "T2";
level = [ "(0,,)" ];
},
{
name = "U10";
level = [ "(0,,)" ];
},
{
name = "V10";
level = [ "(0,,)" ];
}
];
}
...and it seems to find all of these values.
DEBUG 1: Forecast File: WRFMERGE_CAPTEX1_TKEADV.nc
DEBUG 1: Observation File: METOBS_CAPTEX1
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for T2(12,,).
DEBUG 2: Processing masking regions.
DEBUG 2: Processing geography data.
DEBUG 2: For T2(0,,) found 1 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for U10(0,,).
DEBUG 2: For U10(0,,) found 1 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for V10(0,,).
DEBUG 2: For V10(0,,) found 1 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.

For the OBS I successfully readin the temperature, but it does not allow any of my wind data to go through with the following configuration.
obs = {
field = [
{
name = "TMP";
censor_thresh=[>400];
censor_val=[-9999.];
level = "Z2";
},
{
name = "UGRD";
cat_thresh = [>=1.0];
censor_thresh=[>100];
censor_val=[-9999.];
level = "Z10";
},
{
name = "VGRD";
cat_thresh = [>=1.0];
censor_thresh=[>100];
censor_val=[-9999.];
level = "Z10";
}
];

I am getting matched pairs in my temperature and for the most part they look reasonable. On the other hand it is stating that there are no match in the winds.

One thought I had was that WRF outputs the winds as U and V (U10 and V10) with the obs are speed and direction). I tried using changing obs = entry from
{
name = "UGRD";
cat_thresh = [>=1.0];
censor_thresh=[>100];
censor_val=[-9999.];
level = "Z10";
},

to
name = "WIND";
cat_thresh = [>=1.0];
censor_thresh=[>100];
censor_val=[-9999.];
level = "Z10";
},

and that produced a mixed pair including the model's u-comp wind and the observed scalar wind speed (clearly not what I want but proved I could read winds that are there.
.... U10 m_s-1 0,, WIND m/s Z10 ADPSFC FULL NEAREST 1 NA NA NA NA MPR 296 296 SHIP 45.6 -83.5 NA 0 -3.84628 12.34667

So...is my issue at the ascii2nc step where I output the native speed and direction instead a U/V? Is it before that because I am using the raw WRF output (does it need to go through another step)? Or is there something wrong with my configuration at the point_stat level.

Sorry, if this was written excessively long.

Thanks for your help!
Glenn
ascii2nc.out.txt
Ascii2NcConfig_CAPTEX.txt
point_stat_000000L_19830918_120000V.stat.txt
[point_stat_000000L_19830918_120000V_cnt.txt](https:/
point_stat_000000L_19830918_120000V_mpr.txt
/github.com/dtcenter/METplus/files/7341162/point_stat_000000L_19830918_120000V_cnt.txt)
pointstat.out.txt
PointStatConfig_CAPTEX_SFCUV.txt

[robdarvell]

Hi,

This might not answer your question directly, but when we started playing with the wind fields I noted the following in one of my scripts (note these come more from a METplus config file). This came from a GridStat job but may well also fit into a PointStat job.

• For wind data you need to either call things UGRD/VGRD in the variable name
• of the netcdf file or include is_u_wind=true and is_v_wind=true in the _OPTIONS section

So my config looked something like:
OBS_VAR1_NAME = x_wind
OBS_VAR1_LEVELS = "(,)"
OBS_VAR1_OPTIONS = set_attr_level="P100"; is_u_wind=true
(with a corresponding one for the y_wind in our case)

There may well be a cleaner answer to your question (which the DTC people may provide), but thought I show one possible approach.

Rob

[gkhboiler]

Hi Rob,

Thanks for your reply. I think what your answer showed me is that I, being in point space instead of grid space, wasn't thinking about the variable name I was using. UGRD --> U grid (me face palming myself). My observed data is a collection of data points defined by lat-lon and are not gridded). My wind is output as wind speed and wind direction (which explains why it could read the wind speed variable...but then it didn't know how to handle it). So I think my issue is getting to a point where the data can be matched with my model data which is gridded (and why that is reading correctly). Pointstat seems to be handling the scalr temperature issue fine so it seems to be able to match the data point to the appropriate grid point just fine...so next step...how to handle the vector quanity written in a different form in point and grid space.

Anyway, thanks for your reply, it helped me see how what I had here will not work. I think I used UGRD after multiple attempts with other variables (WIND an WDIR) because it at least ran......MET Rookie mistakes! Hopefully there will be an equally...this is what you should have done solution.

Cheers,
Glenn

[CPKalb]

Hi Glenn:

If I am reading things correctly, it seems like you have different variables in your gridded model versus point observations (the u and v wind components in the model and a speed/direction in the observations). I think you want to convert one of these so it matches the other. Do you have the u and v components in your point data? Also, what are the original data formats for your model and point data? It looks like your model data might be in grib/grib2, but what was the original format of the point data?

Tina

[gkhboiler]

Hi Tina,

First, let me preface by saying this is my first attempt at MET so I may be making some basic mistakes. You are correct that my observations and model report the wind differently. My observations reports wind speed and wind direction and my model outputs u- and v- component winds. Our preference is to calculate the statistics with respect to wind speed and wind direction as opposed to component related stats. My observations were originally in little_r format and I used ascii2nc to preprocess for point_stat. From my little run script (metstats.csh) I used the command (file is in my first submission above with added .txt so it would be accepted).
ascii2nc OBS_little_r/metobs_littler_${casename} METOBS_${ucasename} -format little_r -config Ascii2NcConfig_CAPTEX >& ascii2nc.out

My model output is from WRF and is a netCDF file. The WRF output is straight out of the model. When I run point_stat I use the command
point_stat WRFMERGE_CAPTEX1_TKEADV.nc METOBS_${ucasename} PointStatConfig_CAPTEX_SFCUV >& pointstat.out

So it looks like at or by the point_stat level the wind needs to in the same format for the model and the obs and the question will be...can that be done somewhere with the MET software or will I need to do an intermediate step?

Thanks for your help,
Glenn

metstats_cshall.txt
metstats_csh.txt

[CPKalb]

Hi Glenn:

If your data is in grib1 or 2, and the wind variables are named UGRD and VGRD, then MET can derive the wind speed. To use that, you would set the wind variable as
name=“WIND”;
level=“Z10”
When it doesn't find that variable in the file, it would derive wind speed. Unfortunately, this derivation isn't currently available for netCDF data, although it is on our list of features to add. This means some pre-processing will be needed to get the model data to be wind speed and direction. There are a number of ways in which the pre-processing step could be done.

Tina

[gkhboiler]

So would you suggest that I need to add the wind speed and direction to the WRF netCDF output file using something like ncl or python using the U10 and V10 variables?

[gkhboiler]

....though that is probably oversimplifying the answer since the wind speed and wind direction would need to be placed on an unstaggered grid.

[CPKalb]

That is one option. Another option would be to convert your data to grib1/2 if you have that capability. For the staggered grids, MET can perform regridding using the regrid_data_plane tool, or on the fly regridding inside point_stat. Will that work for grid?

[CPKalb]

Hi Glenn:

I'm following up on this question. Will MET's regridding work on your staggered grid? Are there other things I can help with in the conversion from u/v to speed?

Thanks,
Tina

[gkhboiler]

Hi Tina,

I have been working on multiple things at the same time so when this wasn't "low hanging fruit" I completed some other tasks. Looking back at this again it looks like I may need to run a program like WRF_INTERP or p_interp to convert the data from WRF's native Arakawa C-GRID to an unstaggered A-Grid which will get my u- and v-component winds on the same point. It will also help with upper level verification since that can be used to convert data from the Eta vertical coordinate to pressure or height coordinates (p_interp only converts to pressure). I just found these codes so I don't know if I can use it to read in netCDF and output as GRIB or if I need to try to find a conversion for this before or after running these codes. I also discovered the following ticket from 2017 (see below) that talks about using p_interp before running WRF but in this response it suggests to use something called UPP that is no longer supported but still exists at DTC. This response says pinterp doesn't destagger grids but at least in the current documentation it looks like it does.

Phew! So...it seems that my #1 goal is to find theWRF post-processor that best dovetails with the needs of MET. I am hoping to figure this out without compiling too many different post-processors. p_interp looks promising, but I am not sure if this will leave me with u- and v- and I will still be stuck without the wind speed and wind direction and need to add that in as well.

I will continue to look if there is something more current on how best to do this.

Thanks,
Glenn

---

Complete Ticket History

Subject: Point Stat help
From: John Halley Gotway
Time: Wed Feb 08 15:58:50 2017

Matthew,

I see that you have some questions about running the Point-Stat tool.

Regarding post-processing, I'd suggest switching from the pinterp
utility
to using the Unified Post-Processor (UPP). It writes output in GRIB
which
MET can easily handle. Why pinterp output could work fine for precip,
temperature, and dewpoint, MET won't be able to process the winds
since
they're defined on the staggered dimension. I realize that running
UPP,
yet another tool to compile and script up, can be a hassle. But I
think
it'll be less work in the long run.

But as for your specific error from Point-Stat, I suspect that'll be
easy
to fix. In your Point-Stat config file, try using:
name = "T2";
level = [ "(0,,)" ];

I often like plotting the data first:
met-5.2/bin/plot_data_plane
../../../P_INTERP/preproc/ndown/linout/lin/day1/wrfout_d01_2016-08-
12_12:00:00_PLEV

T2.ps
'name="T2"; level="(0,,)";'

Hopefully that'll create a PostScript plot of your data.

Thanks,
John Halley Gotway
met_help at ucar.edu

On Tue, Feb 7, 2017 at 7:27 PM, dawsonmattg at gmail.com via RT <
met_help at ucar.edu> wrote:

Tue Feb 07 19:27:38 2017: Request 79384 was acted upon.
Transaction: Ticket created by dawsonmattg at gmail.com
Queue: met_help
Subject: Point Stat help
Owner: Nobody
Requestors: dawsonmattg at gmail.com
Status: new
Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79384 >

Hello MET help,

    I’m going to go ahead and apologize for all the information

in

this email, but hopefully it will help you better diagnose what the
best
way for me to move forward is. As part of my research I generated 12
different WRF runs over the same week using the same I.C.s and
B.C.s, and
am currently in the process of comparing them against real world
observations to see which config worked the best. I am primary
concerned
with comparing precip, surface temp/dewpoint, and surface winds.

    The first tool I have tried to use has been the point stat

tool. I

am using your met-5.2_bugfix package and did not find any errors
while/after compiling and made sure the test files, like
test_point_stat.sh, all work. The data I am using is as follows:

            Instead of downloading prepbufer files and

converting them

with the PB2NC tool, I just used the UCAR site to do it for me (

http://rda.ucar.edu/datasets/ds337.0/index.html#forms/337_subset.php?_da=y

<http://rda.ucar.edu/datasets/ds337.0/index.html#forms/337_
subset.php?_da=y>)
Request Detail:
Date Limits : 2016-08-12 00:00 2016-08-20 06:00
NCEP Verification Regions : SEC
Parameters : SPFH TMP HGT UGRD VGRD DPT WIND RH MIXR
PRMSL
PREPBUFR Message Types : ADPUPA SFCSHP AIRCAR SPSSMI AIRCFT
QKSWND
SATWND RASSDA PROFLR WDSATR VADWND ASCATW ADPSFC
Quality Mark Threshold : 9
File Compression : gz
Data Output Format : NETCDF

    To convert my WRF data into a readable format I used

P_Interp. I

am now realizing this might not be the best way forward, but seemed
the
easiest to set up as I had to offload all my model output to a
different
cluster that I did not configure WRF on. I have been a bit confused
as
P_Interp says it can unstager data to pressure levels, but on your
website
it says that P_Interp cannot onstager data (at least for use with
MODE and
other tools). Either way do you think I should try use a different
pre-processing software to unpack my WRF data or will P_Interp be
sufficient to compare precip, surface temp/dewpoint and winds?

    When I run my file PointStat.sh using the attached

PointStatConfig1 file, I get the following data error message:

*** Running Point-Stat on prepbuffer files ***
DEBUG 1: Default Config File: /marge/r0/mdawson/met/met-5.2_
bugfix/share/met/config/PointStatConfig_default
DEBUG 1: User Config File: config/PointStatConfig
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=1
DEBUG 1: Forecast File: ../../../P_INTERP/preproc/
ndown/linout/lin/day1/wrfout_d01_2016-08-12_12:00:00_PLEV
DEBUG 1: Observation File: ../data/prepbuffer/12/prepbufr
.gdas.225564.2016081200.nc
DEBUG 2:
DEBUG 2:

---

---

DEBUG 2:
DEBUG 2: Reading data for T2(,,0).
WARNING:
WARNING: PinterpFile::data(NcVar *, const LongArray &, DataPlane &,
double
&) const -> star found in bad slot
WARNING:
ERROR :
ERROR : PinterpFile::valid_time(int) const -> range check error
ERROR :

Looking at the ncdump P_Interp File, T2 (2 meter temperature shows):
float T2(Time, south_north, west_east) ;
T2:FieldType = 104 ;
T2:MemoryOrder = "XY " ;
T2:description = "TEMP at 2 M" ;
T2:units = "K" ;
T2:stagger = "" ;
T2:coordinates = "XLONG XLAT XTIME" ;
T2:missing_value = 1.e+36f ;

I am not sure if the issue is that P_Interp is not properly
extracting
this information, or If I just need to change my PointStatConfig1
file.

Thanks!

Matthew G. Dawson, Capt, USAF
AFIT Student, Civilian Institution Program
Email: Dawsonmattg at gmail.com <mailto:Dawsonmattg at gmail.com>
Cell: (904)-333-5427

[CPKalb]

Hi Glenn:

Were you able to get p_interp or another program working?

Thanks,
Christina

[CPKalb]

Hi Glenn:

We would like to mark this as answered but wanted to be sure that you were able to get this working? Do you need additional information?

Thanks,
Christina

[georgemccabe]

Hi @gkhboiler, I have marked this discussion as answered and locked it to prevent future posts. It sounds like using a tool like p_interp could be used to reformat your data so that it can be read by MET. Please feel free to start a new discussion topic if you continue to have issues reading these data into MET.