5 Models for Nonstationary Time Series

This chapter explores models for time series with nonconstant means, termed nonstationary, contrasting them with deterministic trend models from Chapter 3. It emphasizes stochastic trends, common in economics and business, and introduces differencing as a method to achieve stationarity, leading to ARIMA models.

Introduction to Nonstationarity

A time series is nonstationary if its mean varies over time. Models like $Y_t={\mu }_t+X_t$, where ${\mu }_t$ is a nonconstant mean and $X_t$ is stationary, assume a persistent deterministic trend. However, examples like the random walk (Exhibit 2.1) or monthly oil prices (Exhibit 5.1, January 1986–January 2006) suggest stochastic trends, where apparent trends arise without deterministic components.

Stationarity Through Differencing

Differencing transforms nonstationary series into stationary ones. For an AR(1) model $Y_t=\varphi Y_{t-1}+e_t$:

• $|\varphi |<1$: Stationary (Chapter 4).
• $|\varphi |>1$: Explosive, e.g., $Y_t=3Y_{t-1}+e_t$ yields $Y_t=e_t+3e_{t-1}+3^2{e}_{t-2}+\cdots +3^t{Y}_0$, with exponentially growing variance $\mathrm{Var}(Y_t)=\frac{1}{8}(9^t-1){\sigma }_e^2$ (Exhibit 5.2 simulation).
• \phi = 1: Random walk, $Y_t=Y_{t-1}+e_t$, or $\nabla Y_t=e_t$, where $\nabla Y_t=Y_t-Y_{t-1}$ is stationary white noise.

More generally, if $Y_t=M_t+X_t$ and $M_t$ changes slowly (deterministic or stochastic), $\nabla Y_t$ may be stationary. Example: $Y_t=M_t+e_t$, $M_t=M_{t-1}+{\epsilon }_t$, gives $\nabla Y_t={\epsilon }_t+e_t-e_{t-1}$, an MA(1) process. Second differencing, ${\nabla }^2{Y}_t$, handles linear trends, e.g., $M_t=M_{t-1}+W_t$, $W_t=W_{t-1}+{\epsilon }_t$, yielding ${\nabla }^2{Y}_t={\epsilon }_t+e_t-2e_{t-1}+e_{t-2}$, an MA(2) process.

ARIMA Models

An ARIMA($p,d,q$) model defines $W_t={\nabla }^d{Y}_t$ as a stationary ARMA($p,q$) process, where $d$ is the differencing order (typically 1 or 2). It is defined as:

${\varphi }_p(B)(1-B{)}^d{Y}_t={\theta }_0+{\theta }_q(B){\epsilon }_t$

where:

• $Y_t$: Time series value at time $t$.
• ${\epsilon }_t$: Error term (white noise) at time $t$.
• ${\theta }_0$: Constant term (mean of differenced series).
• $p$: Order of autoregressive terms.
• $d$: Order of differencing.
• $q$: Order of moving average terms.
• $B$: Backshift operator ($B{Y}_t=Y_{t-1}$).
• $(1-B{)}^d$: Differencing operator, applied $d$ times to make series stationary.
• ${\varphi }_p(B)$: Autoregressive operator, ${\varphi }_p(B)=1-{\varphi }_{1}B-{\varphi }_2{B}^2-\cdots -{\varphi }_p{B}^p$.
• ${\theta }_q(B)$: Moving average operator, ${\theta }_q(B)=1+{\theta }_{1}B+{\theta }_2{B}^2+\cdots +{\theta }_q{B}^q$.

The model combines autoregressive (AR), differencing (I), and moving average (MA) components to model stationary or non-stationary time series.

IMA(1,1) Model

$Y_t=Y_{t-1}+e_t-\theta e_{t-1}$:

Property	Expression
Representation	$Y_t=e_t+(1-\theta ){\sum }_{j=1}^{t+m}{e}_{t-j}-\theta e_{-m-1}$
Variance	$\mathrm{Var}(Y_t)=[1+{\theta }^2+(1-\theta {)}^2(t+m)]{\sigma }_e^2$
Correlation	$\mathrm{Corr}(Y_t,Y_{t-k})\approx \sqrt{\frac{t+m-k}{t+m}}\approx 1$ (large $m$, moderate $k$)

Weights persist, reflecting a stochastic trend.

IMA(2,2) Model

${\nabla }^2{Y}_t=e_t-{\theta }_1{e}_{t-1}-{\theta }_2{e}_{t-2}$:

• Representation: $Y_t=e_t+{\sum }_{j=1}^{t+m}[1+{\theta }_2+(1-{\theta }_1-{\theta }_2)j]e_{t-j}+\text{initial terms}$.
• Variance grows rapidly; correlations remain near 1 for moderate lags (Exhibit 5.5 simulation, ${\theta }_1=1$, ${\theta }_2=-0.6$).

ARI(1,1) Model

$Y_t=(1+\varphi )Y_{t-1}-\varphi Y_{t-2}+e_t$, $|\varphi |<1$:

• $\psi$-weights: ${\psi }_k=\frac{1-{\varphi }^{k+1}}{1-\varphi }$, derived via $(1-\varphi x)(1-x)(1+{\psi }_{1}x+\cdots )=1$.

Constant Terms in ARIMA Models

For $W_t={\nabla }^d{Y}_t$ with mean $\mu$, e.g., $W_t={\theta }_0+e_t-\theta e_{t-1}$ in IMA(1,1), $Y_t$ gains a linear trend $(t+m+1){\theta }_0$. Generally, a nonzero $E({\nabla }^d{Y}_t)$ implies a polynomial trend of degree $d$ in $Y_t$.

Other Transformations

Logarithms

If $\sqrt{\mathrm{Var}(Y_t)}={\mu }_t\sigma$, then $\mathrm{Var}(\log Y_t)\approx {\sigma }^2$, stabilizing variance (e.g., oil prices, Exhibit 5.4). For percentage changes $Y_t=(1+X_t)Y_{t-1}$, $\nabla \log Y_t\approx X_t$, often stationary (e.g., electricity data, Exhibits 5.8–5.10).

Power Transformations

Box-Cox transformation: $g(x)=\{\begin{array}{ll}\frac{x^{\lambda }-1}{\lambda },& \lambda \ne 0\\ \log x,& \lambda =0\end{array}$. Applied to positive data, $\lambda$ is estimated (e.g., $\lambda \approx 0$ for electricity data, Exhibit 5.11).

Examples

ARIMA(1,1,1)

To create an ARIMA(1,1,1) model as described in ARIMA Models, we set $p=1$, $d=1$, $q=1$. Below is the step-by-step derivation:

1. Identify parameters:

   • $p=1$: One autoregressive term.
   • $d=1$: One differencing.
   • $q=1$: One moving average term.

2. Substitute into the general formula:

   $${\varphi }_1(B)(1-B{)}^1{Y}_t={\theta }_0+{\theta }_1(B){\epsilon }_t$$

   Plugging in the operators:

   $$(1-{\varphi }_{1}B)(1-B)Y_t={\theta }_0+(1+{\theta }_{1}B){\epsilon }_t$$

3. Expand the left-hand side: Compute $(1-{\varphi }_{1}B)(1-B)$:

   $$(1-{\varphi }_{1}B)(1-B)=1-B-{\varphi }_{1}B+{\varphi }_1{B}^2=1-(1+{\varphi }_1)B+{\varphi }_1{B}^2$$

   So the left-hand side becomes:

   $$[1-(1+{\varphi }_1)B+{\varphi }_1{B}^2]Y_t$$

4. Write the full equation:

   $$[1-(1+{\varphi }_1)B+{\varphi }_1{B}^2]Y_t={\theta }_0+(1+{\theta }_{1}B){\epsilon }_t$$

5. Apply the backshift operator:

   • $B{Y}_t=Y_{t-1}$, $B^2{Y}_t=Y_{t-2}$.
   • Left-hand side: $Y_t-(1+{\varphi }_1)Y_{t-1}+{\varphi }_1{Y}_{t-2}$.
   • Right-hand side: ${\theta }_0+{\epsilon }_t+{\theta }_1{\epsilon }_{t-1}$.

6. Final ARIMA(1,1,1) model:

   $$Y_t-(1+{\varphi }_1)Y_{t-1}+{\varphi }_1{Y}_{t-2}={\theta }_0+{\epsilon }_t+{\theta }_1{\epsilon }_{t-1}$$