Parameter Estimation by Method of Moments

Procedure

Parameter Estimation by Method of Moments equates sample moments with theoretical moments to solve for unknown model parameters.

AR(1)

For AR(1), the theoretical relationship is ${\rho }_1={\varphi }_1$.

Replacing ${\rho }_1$ with sample autocorrelation $r_1$:

${\hat{\varphi }}_1=r_1$

AR(2)

For AR(2), the Yule-Walker equations relate parameters to autocorrelations:

${\rho }_1={\varphi }_1+{\rho }_1{\varphi }_2$ ${\rho }_2={\rho }_1{\varphi }_1+{\varphi }_2$

Replacing ${\rho }_1,{\rho }_2$ with $r_1,r_2$:

$r_1={\hat{\varphi }}_1+r_1{\hat{\varphi }}_2$ $r_2=r_1{\hat{\varphi }}_1+{\hat{\varphi }}_2$

The estimators are:

${\hat{\varphi }}_1=\frac{r_1(1-r_2)}{1-r_1^2}$ ${\hat{\varphi }}_2=\frac{r_2-r_1^2}{1-r_1^2}$

AR(p)

For AR(p), substitute ${\rho }_k$ with $r_k$ in the Yule-Walker equations:

$$\left(\begin{array}{c}{r}_1\\ r_2\\ ⋮\\ r_p\end{array}\right)=\left(\begin{array}{cccc}1& r_1& \cdots & r_{p-1}\\ r_1& 1& \cdots & r_{p-2}\\ ⋮& ⋮& \ddots & ⋮\\ r_{p-1}& r_{p-2}& \cdots & 1\end{array}\right)\left(\begin{array}{c}{\hat{\varphi }}_1\\ {\hat{\varphi }}_2\\ ⋮\\ {\hat{\varphi }}_p\end{array}\right)$$

Solve this linear system to obtain ${\hat{\varphi }}_1,\dots ,{\hat{\varphi }}_p$.

Yule-Walker Estimator

This method is also called the Yule-Walker estimator for AR models.

MA(1)

For MA(1), the theoretical autocorrelation is:

${\rho }_1=\frac{-{\theta }_1}{1+{\theta }_1^2}$

Setting ${\rho }_1=r_1$ yields a quadratic equation in ${\theta }_1$. If $|r_1|<0.5$, there are two real roots:

$\frac{-1\pm \sqrt{1-4r_1^2}}{2r_1}$

The two roots multiply to 1, so only one satisfies the invertibility condition $|{\theta }_1|<1$:

${\hat{\theta }}_1=\frac{-1+\sqrt{1-4r_1^2}}{2r_1}$

Method Fails

• If $|r_1|=0.5$: solution is ${\theta }_1=\pm 1$, not invertible
• If $|r_1|>0.5$: no real solution exists

The method of moments is unreliable for MA models.

ARMA(1,1)

For ARMA(1,1), the autocorrelation function is:

${\rho }_k={\varphi }_1^{k-1}\cdot \frac{(1-{\theta }_1{\varphi }_1)({\varphi }_1-{\theta }_1)}{1-2{\theta }_1{\varphi }_1+{\theta }_1^2},k\ge 1$

Observe that ${\rho }_2={\varphi }_1{\rho }_1$, so:

${\hat{\varphi }}_1=\frac{r_2}{r_1}$

Then use ${\rho }_1$ with $r_1$ and solve the resulting quadratic for ${\hat{\theta }}_1$, keeping only the invertible solution.

Related

• Method of Moments
• Noise Variance Estimation