Autoregressive (AR) Models

When the dependent variable is regressed against one or more lagged values of itself, the resultant model is called as an autoregressive (AR) model. For example, the sales for a firm could be regressed against the sales for the firm in the previous month.

Consider:

In an autoregressive time series, past values of a variable are used to predict the current (and hence future) value of the variable.

Statistical inferences based on ordinary least squares (OLS) estimates for an AR time series model may be invalid unless the time series being modeled is covariance stationary.

A time series is covariance stationary if it satisfies the following three conditions:

1. Constant and finite expected value. The expected value of the time series is constant over time. (Later, we will refer to this value as the mean reverting level.)
2. Constant and finite variance. The time series’ volatility around its mean (i.e., the distribution of the individual observations around the mean) does not change over time.
3. Constant and finite covariance between values at any given lag. The covariance of the time series with leading or lagged values of itself is constant.

The following model illustrates how variable x would be regressed on itself with a lag of one and two periods:

Such a model is referred to as a second order autoregressive model, or an AR(2) model. In general, an AR model of order p, AR(p), is expressed as:

Forecasting With an Autoregressive Model

Autoregressive time series model forecasts are calculated in the same manner as those for other regression models, but since the independent variable is a lagged value of the dependent variable, it is necessary to calculate a one step ahead forecast before a two step ahead forecast can be calculated. The calculation of successive forecasts in this manner is referred to as the chain rule of forecasting.

A one period ahead forecast for an AR(1) model is determined in the following manner:

Likewise, a two step ahead forecast for an AR(1) model is calculated as:

Note that the ^ symbol above the variables in the equations indicates that the inputs used in multi period forecasts are actually forecasts (estimates) themselves. This implies that multi period forecasts are more uncertain than single period forecasts. For example, for a two step ahead forecast, there is the usual uncertainty associated with forecasting $x_{t+1}$ using $x_t$, plus the additional uncertainty of forecasting $x_{t+2}$ using the forecasted value for $x_{t+1}$.

Autocorrelation & Model Fit

When an AR model is correctly specified, the residual terms will not exhibit serial correlation. Serial correlation (or autocorrelation) means the error terms are positively or negatively correlated. When the error terms are correlated, standard errors are unreliable and t tests of individual coefficients can incorrectly show statistical significance or insignificance.

If the residuals have significant autocorrelation, the AR model that produced the residuals is not the best model for the time series being analyzed. The procedure to test whether an AR time series model is correctly specified involves three steps:

A time series exhibits mean reversion if it has a tendency to move toward its mean. In other words, the time series has a tendency to decline when the current value is above the mean and rise when the current value is below the mean. If a time series is at its mean reverting level, the model predicts that the next value of the time series will be the same as its current value (i.e., ${\hat{x}}_t=x_{t-1}$ when a time series is at its mean reverting level).

For an AR(1) model, $x_t$ = $b_0$ + $b_1$$x_{t-1}$, the equality implies that $x_t$ = $b_0$ + $b_1$$x_t$.

Solving for $x_t$, the mean reverting level is expressed as $x_t=\frac{b_0}{1-b_1}$.

So, if $x_t>\frac{b_0}{1-b_1}$, the AR(1) model predicts that $x_t$ + ₁ will be lower than $x_t$, and if $x_t<\frac{b_0}{1-b_1}$, the model predicts that $x_t$ + ₁ will be higher than $x_t$.

In sample forecasts (${\hat{y}}_t$) are within the range of data (i.e., time period) used to estimate the model, which for a time series is known as the sample or test period. In sample forecast errors are ($y_t-{\hat{y}}_t$), where t is an observation within the sample period. In other words, we are comparing how accurate our model is in forecasting the actual data we used to develop the model.

The Predicted vs. Actual Capacity Utilization figure in our Trend Analysis example shows an example of values predicted by the model compared to the values used to generate the model.

Out of sample forecasts are made outside of the sample period. In other words, we compare how accurate a model is in forecasting the y variable value for a time period outside the period used to develop the model. Out of sample forecasts are important because they provide a test of whether the model adequately describes the time series and whether it has relevance (i.e., predictive power) in the real world.

Nonetheless, an analyst should be aware that most published research employs in sample forecasts only.

The root mean squared error (RMSE) criterion is used to compare the accuracy of autoregressive models in forecasting out of sample values. For example, a researcher may have two autoregressive (AR) models: an AR(1) model and an AR(2) model. To determine which model will more accurately forecast future values, we calculate the RMSE (the square root of the average of the squared errors) for the out of sample data.

Note that the model with the lowest RMSE for in sample data may not be the model with the lowest RMSE for out of sample data.

For example, imagine that we have 60 months of historical unemployment data. We estimate both models over the first 36 of 60 months. To determine which model will produce better (i.e., more accurate) forecasts, we then forecast the values for the last 24 of 60 months of historical data.

Using the actual values for the last 24 months as well as the values predicted by the models, we can calculate the RMSE for each model.

The model with the lower RMSE for the out of sample data will have lower forecast error and will be expected to have better predictive power in the future.

In addition to examining the RMSE criteria for a model, we will also want to examine the stability of regression coefficients, which we discuss in the following.

Financial and economic time series inherently exhibit some form of instability or nonstationarity. This is because financial and economic conditions are dynamic, and the estimated regression coefficients in one period may be quite different from those estimated during another period.

Models estimated with shorter time series are usually more stable than those with longer time series because a longer sample period increases the chance that the underlying economic process has changed. Thus, there is a tradeoff between the increased statistical reliability when using longer time periods and the increased stability of the estimates when using shorter periods.

The primary concern when selecting a time series sample period is the underlying economic processes. Have there been regulatory changes? Has there been a dramatic change in the underlying economic environment?

If the answer is yes, then the historical data may not provide a reliable model. Merely examining the significance of the autocorrelation of the residuals will not indicate whether the model is valid. We must also examine whether the data is covariance stationary.