2.5 The Correlation Coefficient

Definition: Covariance

Let $(X,Y)$ have a joint distribution with means ${\mu }_1,{\mu }_2$ and variances ${\sigma }_1^2,{\sigma }_2^2$.

The covariance of $(X,Y)$ is defined as:

$$\begin{array}{rl}\text{Cov}(X,Y)& =E[(X-{\mu }_1)(Y-{\mu }_2)]\\ & =E[XY]-{\mu }_1{\mu }_2\end{array}$$

Definition: Correlation Coefficient

If ${\sigma }_1>0$ and ${\sigma }_2>0$, the correlation coefficient between $X$ and $Y$ is:

$\rho =\frac{E[(X-{\mu }_1)(Y-{\mu }_2)]}{{\sigma }_1{\sigma }_2}=\frac{\mathrm{Cov}(X,Y)}{{\sigma }_1{\sigma }_2}$

Correlatino measures the linear dependence between 2 random variables.

Properties

Key Properties

• $E(XY)={\mu }_1{\mu }_2+\mathrm{cov}(X,Y)={\mu }_1{\mu }_2+\rho {\sigma }_1{\sigma }_2$
• $-1\le \rho \le 1$ (Theorem 2.5.1)
• $\mathrm{cov}(X,Y)=0$ when $X\perp Y$ (Theorem 2.5.2)
• $\rho =0$ does not imply independence (Example 2.5.3)

Theorem 2.5.3: Linear Conditional Mean

If $E(Y|X)$ is linear in $X$, then:

$E(Y|X)={\mu }_2+\rho \frac{{\sigma }_2}{{\sigma }_1}(X-{\mu }_1)$

And the conditional variance:

$\mathrm{Var}(Y|X)={\sigma }_2^2(1-{\rho }^2)$

See also

Cheatsheets:

• Covariance
• Correlation Coefficient

Examples

Discrete Case (Example 2.5.1)

For coin flips: $X_1$ = heads on first two flips, $X_2$ = heads on all three flips.

• $E(X_1)=1$, $E(X_2)=3/2$
• ${\sigma }_1^2=1/2$, ${\sigma }_2^2=3/4$
• $\mathrm{cov}(X_1,X_2)=1/2$
• $\rho =0.816$

Continuous Case (Example 2.5.2)

$f(x,y)=x+y$, $0<x<1$, $0<y<1$:

• ${\mu }_1={\mu }_2=7/12$
• ${\sigma }_1^2={\sigma }_2^2=11/144$
• $\mathrm{cov}(X,Y)=-1/144$
• $\rho =-1/11$