Hyndsight
1,478 FOLLOWERS
A blog by Rob J Hyndman. Rob is a Professor of Statistics in the Department of Econometrics and Business Statistics at Monash University, and Editor-in-Chief of the International Journal of Forecasting. He develops statistical models, algorithms and software for forecasting, analysing and visualizing data.
Hyndsight
3w ago
Forecasting interrupted time series data is a major challenge for forecasting teams, especially in light of events such as the COVID-19 pandemic. This paper investigates several strategies for dealing with interruptions in time series forecasting, including highly adaptable models, intervention models, marking interrupted periods as missing, forecasting what may have been, downweighting the interruption period, and ensemble models. Each approach offers specific advantages and disadvantages, such as adaptability, memory retention, data integrity, flexibility, and accuracy. We evaluate the effec ..read more
Hyndsight
3M ago
Collections of time series formed via aggregation are prevalent in many fields. These are commonly referred to as hierarchical time series and may be constructed cross-sectionally across different variables, temporally by aggregating a single series at different frequencies, or even generalised beyond aggregation as time series that respect linear constraints. When forecasting such time series, a desirable condition is for forecasts to be coherent: to respect the constraints. The past decades have seen substantial growth in this field with the development of reconciliation methods that ensure ..read more
Hyndsight
4M ago
Abstract
Forecast reconciliation is a post-forecasting process that involves transforming a set of incoherent forecasts into coherent forecasts which satisfy a given set of linear constraints for a multivariate time series. We extend the current state-of-the-art cross-sectional probabilistic forecast reconciliation approach to encompass a cross-temporal framework, where temporal constraints are also applied. Our proposed methodology employs both parametric Gaussian and non-parametric bootstrap approaches to draw samples from an incoherent cross-temporal distribution. To improve the estimation ..read more
Hyndsight
5M ago
The AIC returned by TSLM() is different from that returned by lm(). Why?
I get this question a lot, so I thought it might help to explain some issues with AIC calculation.
First, the equation for the AIC is given by where is the likelihood of the model and is the number of parameters that are estimated (including the error variance). For a linear regression model with iid errors, fitted to observations, the log-likelihood can be written as where is the residual for the th observation. The AIC is then Since we don’t know , we estimate it using the mean squared error (the maximum likelih ..read more
Hyndsight
5M ago
Does it make any sense to compute p-values for prediction intervals?
I received this email today:
My team recently used some techniques found in your writings to perform forecasts … Our work has been well received by reviewers, but one commenter asked two questions that I was hoping you may be able to provide insight on.
First, they wanted to know if we could provide P-values for our prediction intervals. In our work, we said, “Observed rates were deemed significantly different from expected rates when they did not fall within the 95% PI.” This same language has been used by others publishe ..read more
Hyndsight
6M ago
Features of time series are useful in identifying suitable models for forecasting. We present a general framework, labelled FFORMS (Feature-based FORecast Model Selection), which selects forecast models based on features calculated from each time series. The FFORMS framework builds a mapping that relates the features of a time series to the “best” forecast model using a classification algorithm such as a random forest. The framework is evaluated using time series from the M-forecasting competitions and is shown to yield forecasts that are almost as accurate as state-of-the-art methods, but are ..read more
Hyndsight
6M ago
I’ve had several requests for an R function to simulate future values from a TBATS model. We will eventually include TBATS in the fable package, and the facilities will be added there. But in the meantime, if you are using the forecast package and want to simulate from a fitted TBATS model, here is how do it.
Simulating via one-step forecasts
Doing it efficiently would require a more complicated approach, but this is super easy if you are willing to sacrifice some speed. The trick is to realise that a simulation can be handled easily for almost any time series model using residuals and one-ste ..read more
Hyndsight
6M ago
I am writing a new textbook on anomaly detection. It probably won’t be finished for at least a year, but here is an excerpt.
There is a widespread myth that NASA did not discover the hole in the ozone layer above the Antarctic until some British scientists spotted it in 1985, because they had been throwing away anomalous data that would have revealed it. This is not entirely true, but the real story is also instructive (Pukelsheim 1990; Christie 2001, 2004).
The British Antarctic Survey had collected data at the Halley Research Station, on the edge of the Brunt Ice Shelf in Antarctica, since 1 ..read more