Forecast evaluation and ensemble building

An important practical goal of the forecast hub is the comparative study and combination of different forecasts.

Data driven combination of different forecasts into a so-called ensemble forecast is a standard method from meteorology and increasingly adopted in epidemic forecasting (see eg this paper by Reich et al). We will apply forecast combination methods of different complexities, which will be outlined here as the project advances.

The idea of combining forecasts in a data driven way is intimately linked to the need to quantify forecast performance. After all, an optimal forecast combination strategy needs to be identified according to a well-defined criterion. This also involves evaluating (or scoring) the different forecasts submitted to our system, but we want to highlight that this is not a competition. As forecasts are reported as predictive quantiles, we are applying evaluation methods which are designed specifically for this format. Details can be found in this preprint.