Conditional Least Squares

Procedure

Conditional Least Squares minimizes the conditional sum of squares function to estimate model parameters, treating the model as a regression problem.

AR(1)

Step 1: Estimate $\mu$

Set $\frac{\partial S_c}{\partial \mu }=0$, yielding:

$\hat{\mu }=\frac{1}{n-1}{\sum }_{t=2}^n{Y}_t-\varphi {\sum }_{t=2}^n{Y}_{t-1}$

For large $n$ with stationary process:

$\hat{\mu }\approx \bar{Y}$

Step 2: Estimate $\varphi$

Set $\frac{\partial S_c}{\partial \varphi }=0$ with $\hat{\mu }=\bar{Y}$:

$\hat{\varphi }=\frac{{\sum }_{t=2}^n(Y_t-\bar{Y})(Y_{t-1}-\bar{Y})}{{\sum }_{t=2}^n(Y_{t-1}-\bar{Y}{)}^2}$

Comparison with $r_1$

This is similar to sample autocorrelation $r_1$, but the denominator is missing one term $(Y_n-\bar{Y}{)}^2$. For stationary processes with large $n$, the difference is negligible.

AR(2)

Extend AR(1) approach:

$\hat{\mu }\approx \bar{Y}$

For ${\varphi }_1,{\varphi }_2$, solve the sample Yule-Walker equations:

$$\{\begin{array}{l}{r}_1={\hat{\varphi }}_1+r_1{\hat{\varphi }}_2\\ r_2=r_1{\hat{\varphi }}_1+{\hat{\varphi }}_2\end{array}$$

AR(p)

Minimizing $S_c$ yields the same system as the sample Yule-Walker equations.

Connection

For stationary AR(p), conditional least squares and method of moments (Yule-Walker) produce nearly identical estimates for large samples.

MA(1)

For MA(1) model $Y_t=e_t-\theta e_{t-1}$, use the invertible AR representation:

$Y_t=-\theta Y_{t-1}-{\theta }^2{Y}_{t-2}-\cdots +e_t$

The conditional sum of squares becomes:

$S_c(\theta )={\sum }_{t=2}^n[Y_t+\theta Y_{t-1}+{\theta }^2{Y}_{t-2}+\cdots {]}^2$

Nonlinear Optimization Required

$S_c(\theta )$ is nonlinear in $\theta$. No explicit solution exists. Use numerical methods.

Mixed Models (ARMA)

For ARMA(p,q), the conditional sum of squares involves both AR and MA components:

$e_t=Y_t-\mu -{\varphi }_1(Y_{t-1}-\mu )-\cdots -{\varphi }_p(Y_{t-p}-\mu )+{\theta }_1{e}_{t-1}+\cdots +{\theta }_q{e}_{t-q}$

Model Fitting Procedure

1. Set initial values: $e_{p+1-q}=e_{p-q}=\cdots =e_0=0$
2. Compute errors recursively: $e_t=Y_t-\hat{\mu }-\sum {\hat{\varphi }}_i{Y}_{t-i}+\sum {\hat{\theta }}_j{e}_{t-j}$
3. Minimize $S_c=\sum e_t^2$ numerically

Related

• Conditional Sum of Squares Function
• Least Square Method
• Unconditional Least Squares