<style> body{ text-align: justify; } </style>

harpIO

{harpIO} provides a set of read and write functions for harp plus the option to transform gridded data to point data, cross sections or other grids at read time. {harpIO} is able to read grib1, grib2, FA, NetCDF and vfld files from weather forecast / climate models; station observations from vobs files, radar / satellite observations from HDF5 files and in the future from in-house databases at selected meteorological institutes. Some of these formats require support packages to be installed and you will be given advice on which packages to install when you attempt to read those data.

Installation

install.packages("remotes")
remotes::install_github("harphub/harpIO")

System libraries

{harpIO} uses the {meteogrid} package for interpolation and the geogrid class for storing gridded data. This package makes use of the PROJ4 library for handling projections. If you do not have PROJ4 installed you can install with:

sudo apt-get install libproj-dev

If you do not have sudo rights, speak to your system administrator.

If you have the PROJ libraries installed in a standard location (e.g. /usr/local) meteogrid will install without problems. However, if the PROJ libraries are in a non standard location, you need to tell the install function where they are:

remotes::install_github(
  "harphub/harpIO",
  configure.args = c(
    meteogrid = "--with-proj-lib=/path/to/proj/lib --with-proj-include=/path/to/proj/include"
  )
)

Alternatively you can set environment variables

export PROJ_LIBS=/path/to/proj/lib
export PROJ_INCLUDE=/path/to/proj/include

If you include these environment variables in your .bashrc file, or equivalent, you won’t need to worry about it when you wish to install an update to meteogrid.

Or you can set compiler and linker options in the file $HOME/.R/Makevars

CPPFLAGS=-I/path/to/proj/include
LDFLAGS=-L/path/to/proj/lib -Wl,-rpath,/path/to/proj/lib

In this case you only have to set them once and not worry about it when you wish to install an update to meteogrid.

When setting environment variables or creating a Makevars file, R must be restarted for the changes to take effect before running remotes::install_github("harphub/harpIO").

Reading, transforming and writing forecast data

The main workhorse for {harpIO} is read_forecast(). This function can read multiple files from multiple sources of both gridded and point data. See the articles Reading forcast model data and Options for file formats for more information.

For gridded data you have the option to transform the data to point data, another grid, or a vertical cross section. This is done with the transformation and transformation_opts arguments. Interpolation to points is done by providing a data frame of latitiude and longitude locations, regridding is done by providing a definition of the grid to which to regrid (and if necessary reproject) the data, and a cross section is obtained by providing the latitude and longitude of the end points and reading 3 dimensional data. See the article Transforming forecast model data for more information.

When data are interpolated to points it is possible to save the result to SQLite files. These are portable database files that allow fast access to the data with the ability to select and filter what you wish to read. You can tell read_forecast() to output point data to SQLite files by setting the argument output_file_opts = sqlite_opts(path = "/path/to/output"). These data can then be read with read_point_forecast(). For gridded data there is generally no advantage to outputting to another format, so this option is only available if data are either point data to begin with, or transformation = "interpolate".

Reading observation data

For point observations, {harpIO} can currently only read from vobs files, as produced by the HIRLAM consortium, and write them out to SQLite files using read_obs_convert() for faster access. The SQLite files can then be read with read_point_obs(). For gridded observations, data can be read from NetCDF, grib and some HDF5 files using read_grid(). A new function, read_obs(), is currently under development to harmonise the reading and transforming of observations in the same way that forecast data are handled.