Common Expectation and Variance Operations

Notation

• $X,Y,Z,X_i$: Random variables
• $a,b,c,c_i,a_i$: Constants/scalars
• $X\perp Y$: $X$ and $Y$ are independent
• $\text{Cov}(X,Y)$: Covariance between $X$ and $Y$
• $\text{Corr}(X,Y)$: Correlation between $X$ and $Y$

Basic Operations

$$\begin{array}{rl}E[c]& =c\\ E[X\pm c]& =E[X]\pm c\\ E[cX]& =cE[X]\\ & \\ \text{Var}(X\pm c)& =\text{Var}(X)\\ \text{Var}(cX)& =c^2\text{Var}(X)\end{array}$$

Multiple Random Variables

$$\begin{array}{rl}E[X\pm Y]& =E[X]\pm E[Y]\\ E[aX+bY]& =aE[X]+bE[Y]\\ & \\ E[XY]& =E[X]E[Y]\text{when }X\perp Y\\ & \\ \text{Var}(X\pm Y)& =\text{Var}(X)+\text{Var}(Y)\pm 2\text{Cov}(X,Y)\\ \text{Var}(aX+bY)& =a^2\text{Var}(X)+b^2\text{Var}(Y)+2ab\text{Cov}(X,Y)\end{array}$$

Tip

Recall $\text{Cov}(X,Y)=0$ if $X\perp Y$

Sum of Variables

$$\begin{array}{rl}E\left[\sum _{i=1}^n{X}_i\right]& =\sum _{i=1}^{n}E[X_i]\\ E\left[\sum _{i=1}^n{c}_i{X}_i\right]& =\sum _{i=1}^n{c}_{i}E[X_i]\\ & \\ \text{Var}\left(\sum _{i=1}^n{a}_i{X}_i\right)& =\sum _{i=1}^n{a}_i^2\text{Var}(X_i)+2\sum _{i<j}{a}_i{a}_j\text{Cov}(X_i,X_j)\\ \text{Var}\left(\sum _{i=1}^n{c}_i{X}_i\right)& =\sum _{i=1}^n{c}_i^2\text{Var}(X_i)\text{when all }{X}_i\text{ are independent}\end{array}$$

Special Formulas

$$\begin{array}{rl}\boxed{\text{Var}(X)}& =E[X^2]-E[X{]}^2\\ & =E[(X-E[X]{)}^2]\end{array}$$

Covariance

$$\begin{array}{rl}\boxed{\text{Cov}(X,Y)}& =E[(X-E[X])(Y-E[Y])]\\ & =E[XY]-E[X]E[Y]\end{array}$$ $$\begin{array}{rl}\text{Cov}(X,X)& =\text{Var}(X)\\ \text{Cov}(X,Y)& =\text{Cov}(Y,X)\\ \text{Cov}(X,Y)& =0\text{when }X\perp Y\\ & \\ \text{Cov}(X\pm Y,Z)& =\text{Cov}(X,Z)\pm \text{Cov}(Y,Z)\\ \text{Cov}(aX,bY)& =ab\text{Cov}(X,Y)\\ \text{Cov}(X+a,Y+b)& =\text{Cov}(X,Y)\end{array}$$

Correlation Coefficient

$$\text{Corr}(X,Y)=\frac{\text{Cov}(X,Y)}{\sqrt{\text{Var}(X)}\sqrt{\text{Var}(Y)}}$$

Properties

$$\begin{array}{rl}-1\le \text{Corr}(X,Y)& \le 1\\ \text{Corr}(X,Y)& =0\text{when }X\perp Y\\ \text{Corr}(X,Y)& =\pm 1⟹Y=a\pm bX\text{ (linear relationship)}\end{array}$$