forked from andrew-MET/harpIO
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.Rmd
78 lines (58 loc) · 4.96 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
<style>
body{
text-align: justify;
}
</style>
```{r setup, include = FALSE}
knitr::opts_chunk$set(
collapse = FALSE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# harpIO <a href=#><img src='man/figures/harp_logo_dark.svg' align="right" height="131.5" style="margin-left:30px" /></a>
{harpIO} provides a set of read and write functions for harp plus the option to transform gridded data to point data, cross sections or other grids at read time. {harpIO} is able to read _grib1_, _grib2_, _FA_, _NetCDF_ and _vfld_ files from weather forecast / climate models; station observations from _vobs_ files, radar / satellite observations from _HDF5_ files and in the future from in-house databases at selected meteorological institutes. Some of these formats require support packages to be installed and you will be given advice on which packages to install when you attempt to read those data.
## Installation
```{r eval=FALSE}
install.packages("remotes")
remotes::install_github("harphub/harpIO")
```
### System libraries
{harpIO} uses the [{meteogrid}](https://github.com/harphub/meteogrid) package for interpolation and the geogrid class for storing gridded data. This package makes use of the [PROJ4](https://proj4.org) library for handling projections. If you do not have PROJ4 installed you can install with:
```{bash eval=FALSE}
sudo apt-get install libproj-dev
```
If you do not have sudo rights, speak to your system administrator.
If you have the PROJ libraries installed in a standard location (e.g. /usr/local) meteogrid will install without problems. However, if the PROJ libraries are in a non standard location, you need to tell the install function where they are:
```{r eval=FALSE}
remotes::install_github(
"harphub/harpIO",
configure.args = c(
meteogrid = "--with-proj-lib=/path/to/proj/lib --with-proj-include=/path/to/proj/include"
)
)
```
Alternatively you can set environment variables
```{bash eval=FALSE}
export PROJ_LIBS=/path/to/proj/lib
export PROJ_INCLUDE=/path/to/proj/include
```
If you include these environment variables in your .bashrc file, or equivalent, you won't need to worry about it when you wish to install an update to meteogrid.
Or you can set compiler and linker options in the file $HOME/.R/Makevars
```{bash eval=FALSE}
CPPFLAGS=-I/path/to/proj/include
LDFLAGS=-L/path/to/proj/lib -Wl,-rpath,/path/to/proj/lib
```
In this case you only have to set them once and not worry about it when you wish to install an update to meteogrid.
When setting environment variables or creating a Makevars file, R must be restarted for the changes to take effect before running `remotes::install_github("harphub/harpIO")`.
## Reading, transforming and writing forecast data
The main workhorse for {harpIO} is `read_forecast()`. This function can read multiple files from multiple sources of both gridded and point data. See the articles [Reading forcast model data](articles/read_raw_forecast.html) and [Options for file formats](articles/file_options.html) for more information.
For gridded data you have the option to transform the data to point data, another grid, or a vertical cross section. This is done with the `transformation` and `transformation_opts` arguments. Interpolation to points is done by providing a data frame of latitiude and longitude locations, regridding is done by providing a definition of the grid to which to regrid (and if necessary reproject) the data, and a cross section is obtained by providing the latitude and longitude of the end points and reading 3 dimensional data. See the article [Transforming forecast model data](articles/transformations.html) for more information.
When data are interpolated to points it is possible to save the result to [SQLite](https://www.sqlite.org/) files. These are portable database files that allow fast access to the data with the ability to select and filter what you wish to read. You can tell `read_forecast()` to output point data to SQLite files by setting the argument `output_file_opts = sqlite_opts(path = "/path/to/output")`. These data can then be read with `read_point_forecast()`. For gridded data there is generally no advantage to outputting to another format, so this option is only available if data are either point data to begin with, or `transformation = "interpolate"`.
## Reading observation data
For point observations, {harpIO} can currently only read from _vobs_ files, as produced by the [HIRLAM](http://hirlam.org/index.php/hirlam-programme-53) consortium, and write them out to SQLite files using `read_obs_convert()` for faster access. The SQLite files can then be read with `read_point_obs()`. For gridded observations, data can be read from _NetCDF_, _grib_ and some _HDF5_ files using `read_grid()`. A new function, `read_obs()`, is currently under development to harmonise the reading and transforming of observations in the same way that forecast data are handled.