Timeseries forecasting examples?

[langestefan]

Hi, I am looking for some examples of timeseries forecasting with RxInfer.

I found the Predicting Bike Rental Demand example, but other than that I haven't found any. Research papers are also welcome.

[albertpod]

Hi @langestefan,

Unfortunately, there is no good example indeed.

You can take a look at this discussion as I actually was doing latent multivariate AR prediction by supplying missing to the data.

Can you share a bit more what you are aiming to do?

[langestefan]

Thanks, I replied below to @bvdmitri.

[bvdmitri]

Yeah, perhaps we should write a dedicated section for it as we have only some scattered examples:

• https://reactivebayes.github.io/RxInferExamples.jl/dev/examples/Basic%20Examples/Kalman%20filtering%20and%20smoothing/Kalman%20filtering%20and%20smoothing/#Handling-Missing-Data
• https://reactivebayes.github.io/RxInfer.jl/stable/manuals/inference/static/#Missing-data-points-and-predictions
• the one that you mentioned

The idea here is pretty simple, you can create your model longer than you actually have data. Consider a vector of observations y of length 110, and you want to forecast 10 extra points. For those 10 last "observations" you simply put missing as an observation in y such that it becomes of length 110. RxInfer should figure it out automatically from the type of the vector Vector{Union{Float64, Missing}} and create an extra field result.predictions where you can check the predictions for the last 10 elements. Or check posteriors for states directly. Here is a small demonstration using a very simplistic random walk model (thus a simple forecasting result), but the idea stays the same for other models:

using RxInfer, Plots, Random

function generate_data(n)
    rng = MersenneTwister(42)
    a = 1.1
    x = [ randn(rng) ]
    y = [ rand(NormalMeanPrecision(x[1], 1.0)) ]
    for i in 2:n
        push!(x, rand(NormalMeanPrecision(x[i - 1] + a, 0.1)))
        push!(y, rand(NormalMeanPrecision(x[i], 0.01)))
    end
    return x, y
end

x, y = generate_data(100)

@model function random_walk(y)
    x[1] ~ Normal(mean = 0.0, precision = 1.0)
    y[1] ~ Normal(mean = x[1], precision = 1.0)
    for i in 2:length(y)
        x[i] ~ Normal(mean = x[i - 1] + 1.1, precision = 0.1)
        y[i] ~ Normal(mean = x[i], precision = 0.1)
    end
end

extend_y = [ y..., [ missing for i in 1:10 ]... ]
results = infer(
    model = random_walk(),
    data = (y = extend_y, )
)

plot(mean.(results.posteriors[:x]), ribbon = std.(results.posteriors[:x]))
scatter!(y)

So if you already have a model that runs, ideally you should be able to add missings to your observations vector and it would hopefully "just work"

[langestefan]

Yeah, perhaps we should write a dedicated section for it as we have only some scattered examples

Definitely. I like how JuMP has organized its examples, everything neatly grouped by problem class. Perhaps RxInfer could for example have groups for forecasting, parameter estimation, filtering, control etc.

Here is a small demonstration using a very simplistic random walk model (thus a simple forecasting result), but the idea stays the same for other models

Great example, thanks. I like how state estimation can be naturally extended to forecasting in the Bayesian framework. I guess that should also work with tabular data like this. In my specific application power is unknown and has to be forecasted, but the other variables can be obtained from weather forecasts.

Timestamp	Temperature (°C)	Wind Speed (m/s)	Humidity (%)	Power Output (kW)
2025-01-22 02:00	14.2	8.0	68	200
2025-01-22 03:00	13.7	7.8	65	180
2025-01-22 04:00	13.1	7.5	60	160
2025-01-22 05:00	12.5	7.0	58	Missing
2025-01-22 06:00	12.0	6.8	55	Missing
2025-01-22 07:00	11.6	6.4	53	Missing

I noticed you didn't use UnfactorizedData in this example, would that make it more accurate?

[bvdmitri]

If you have a working probabilistic generative model that models the hidden state of the weather and uses Temperature etc with Power Output as observations then yes it can be forecasted. Perhaps you can look at Linear Gaussian Multivariate Dynamical System that models multivariate observations and automatically infers the state transition matrix. The state transition matrix can also be estimated with ContinousTransition node (tagging author @albertpod)

When writing this as a response I realized that something like this

y = [
    [ 14.2, 8.0, 68, 200 ],
    [ 11.6, 6.4, 53, missing ]
]

will not work unfortunately.

The variables that contain missing values should be observed separate. This should work:

y_temp = [ 14.2, 8.0 ]
y_wind = [ 8.0, 6.4 ]
y_humid = [ 68, 53 ]
y_power = [ 200, missing ]

or

y_temp_wind_humid = [
    [ 14.2, 8.0, 68 ],
    [ 11.6, 6.4, 53 ]
]
y_power = [
    200,
    missing
]

or

y = [
    [ 14.2, 8.0, 68, 200 ],
    missing # the entire observation must be missing then, which may not be the best option
]

We may eventually support mixed observations, but currently it is not supported.

UnfactorizedData should indeed improve the estimates, though it may break Free Energy computation (if its relevant) thus it is not used by default.

[albertpod]

@langestefan, do you mean that you observe Temperature, Wind Speed, and Humidity (TWH), and are trying to predict the Power Output? This might require a different modeling approach compared to a typical state-space example, as you have predictors here, TWH, which you don't necessarily need to model. Or do you want to model the dynamics of those variables as well? Either way, this sounds like an interesting example!

[langestefan]

@albertpod This was just some dummy data to show how I would use it.

This is one of the real applications I'm looking at, power forecasting for wind power farms: Predico: collaborative forecasting platform

Most solutions I've seen do some kind of frequency domain decomposition and dump the frequency features in some kind of neural network architecture. And then they use heuristics to compute the confidence intervals. I think we can do better than that :)

This might require a different modeling approach compared to a typical state-space example, as you have predictors here, TWH, which you don't necessarily need to model. Or do you want to model the dynamics of those variables as well?

Yes correct, this will likely need some kind of neural network as it very computationally expensive to model a wind farm.

[albertpod]

We can certainly do better than that, but we can also combine NNs within model macro. How many dimensions do people usually deal with when modeling a wind farm?