Continuous Distributions

Summary

Distribution	pdf	Mean	Variance	mgf
Uniform $U(a,b)$	$\frac{1}{b-a}$	$\frac{a+b}{2}$	$\frac{(b-a{)}^2}{12}$	$\frac{e^{bt}-e^{at}}{t(b-a)}$
Gamma $\Gamma (\alpha ,\beta )$	$\frac{1}{\Gamma (\alpha ){\beta }^{\alpha }}{x}^{\alpha -1}{e}^{-x/\beta }$	$\alpha \beta$	$\alpha {\beta }^2$	$(1-\beta t{)}^{-\alpha }$
Exponential $\text{Exp}(\lambda )$ $\Gamma \left(1,\frac{1}{\lambda }\right)$	$\lambda e^{-\lambda x}$	$\frac{1}{\lambda }$	$\frac{1}{{\lambda }^2}$	$\frac{\lambda }{\lambda -t}$
Chi-square ${\chi }^2(r)$ $\Gamma \left(\frac{r}{2},2\right)$	$\frac{1}{\Gamma (r/2)2^{r/2}}{x}^{(r/2)-1}{e}^{-x/2}$	$r$	$2r$	$(1-2t{)}^{-r/2}$
Normal $N(\mu ,{\sigma }^2)$	$\frac{1}{\sigma \sqrt{2\pi }}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)$	$\mu$	${\sigma }^2$	$\exp \left(\mu t+\frac{{\sigma }^2{t}^2}{2}\right)$
Standard Normal $Z\sim N(0,1)$	$\frac{1}{\sqrt{2\pi }}\exp \left(-\frac{x^2}{2}\right)$	$0$	$1$	$\exp \left(\frac{t^2}{2}\right)$
t-distribution $t(\nu )$	$\frac{\Gamma \left(\frac{\nu +1}{2}\right)}{\Gamma \left(\frac{\nu }{2}\right)\sqrt{\nu \pi }}{\left(1+\frac{x^2}{\nu }\right)}^{-\frac{\nu +1}{2}}$	$0$ (for $\nu >1$)	$\frac{\nu }{\nu -2}$ (for $\nu >2$)	Does not exist
F-distribution $F({\nu }_1,{\nu }_2)$	Here	$\frac{{\nu }_2}{{\nu }_2-2}$ (for ${\nu }_2>2$)	Here	Does not exist
Bivariate Normal $N_2({\mu }_1,{\mu }_2,{\sigma }_1^2,{\sigma }_2^2,\rho )$	Here	$({\mu }_1,{\mu }_2)$	$({\sigma }_1^2,{\sigma }_2^2)$	Here

Uniform distribution

$X\sim U(a,b)$

All outcomes in an interval are equally likely

• pdf: $\frac{1}{b-a}$, $a<x<b$
• mean: $\frac{a+b}{2}$
• var: $\frac{(b-a{)}^2}{12}$
• mgf: $\frac{e^{bt}-e^{at}}{t(b-a)}$, $t\ne 0$

Gamma distribution

$X\sim \Gamma (\alpha ,\beta )$

Commonly used to model waiting time until an event occurs

• pdf: $\frac{1}{\Gamma (\alpha ){\beta }^{\alpha }}{x}^{\alpha -1}{e}^{-x/\beta }$, $x>0$, $\alpha >0$, $\beta >0$
• mean: $\alpha \beta$
• var: $\alpha {\beta }^2$
• mgf: $(1-\beta t{)}^{-\alpha }$, $t<\frac{1}{\beta }$

Gamma function $\Gamma (\alpha )=(\alpha -1)\Gamma (\alpha -1)={\int }_0^{\infty }{y}^{\alpha -1}{e}^{-y},dy$ For positive integers: $\Gamma (n)=(n-1)!$

Exponential distribution

$X\sim \text{Exp}(\lambda )$

Models time between events in a Poisson process. Special case of Gamma distribution with $\alpha =1$, $\beta =\frac{1}{\lambda }$.

• pdf: $\lambda e^{-\lambda x}$, $x>0$, $\lambda >0$
• mean: $\frac{1}{\lambda }$
• var: $\frac{1}{{\lambda }^2}$
• mgf: $\frac{\lambda }{\lambda -t}$, $t<\lambda$

Chi-square distribution

$X\sim {\chi }^2(r)$

Special case of Gamma distribution with $\alpha =\frac{r}{2}$, $\beta =2$. Used in hypothesis testing and confidence intervals.

• pdf: $\frac{1}{\Gamma (r/2)2^{r/2}}{x}^{(r/2)-1}{e}^{-x/2}$, $x>0$, $r>0$
• mean: $r$
• var: $2r$
• mgf: $(1-2t{)}^{-r/2}$, $t<\frac{1}{2}$

Normal distribution

$X\sim N(\mu ,{\sigma }^2)$

The most important continuous distribution, appears naturally due to Central Limit Theorem

• pdf: $\frac{1}{\sigma \sqrt{2\pi }}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)$, $x\in \mathbb{R}$
• mean: $\mu$
• var: ${\sigma }^2$
• mgf: $\exp \left(\mu t+\frac{{\sigma }^2{t}^2}{2}\right)$

Standardization: If $X\sim N(\mu ,{\sigma }^2)$, then $Z=\frac{X-\mu }{\sigma }\sim N(0,1)$

Relationship to Chi-square: If $X\sim N(\mu ,{\sigma }^2)$, then ${\left(\frac{X-\mu }{\sigma }\right)}^2\sim {\chi }^2(1)$

Z-distribution

$Z\sim N(0,1)$

The standardized version of the normal distribution. Fundamental in statistical theory and hypothesis testing.

• pdf: $\frac{1}{\sqrt{2\pi }}\exp \left(-\frac{x^2}{2}\right)$, $x\in \mathbb{R}$
• mean: $0$
• var: $1$
• mgf: $\exp \left(\frac{t^2}{2}\right)$

Properties:

• $P(Z\le z)=\Phi (z)$ where $\Phi$ is the standard normal CDF
• $P(Z\le -z)=1-P(Z\le z)$ (symmetry)
• $P(-z\le Z\le z)=2\Phi (z)-1$

t-distribution

$X\sim t(\nu )$

Student’s t-distribution with $\nu$ degrees of freedom. Used when population variance is unknown and estimated from sample data.

• pdf: $\frac{\Gamma \left(\frac{\nu +1}{2}\right)}{\Gamma \left(\frac{\nu }{2}\right)\sqrt{\nu \pi }}{\left(1+\frac{x^2}{\nu }\right)}^{-\frac{\nu +1}{2}}$, $x\in \mathbb{R}$, $\nu >0$
• mean: $0$ (for $\nu >1$)
• var: $\frac{\nu }{\nu -2}$ (for $\nu >2$)
• mgf: Does not exist

Key properties:

• Symmetric around 0, bell-shaped like normal distribution
• Heavier tails than standard normal
• As $\nu \to \infty$, $t(\nu )\to N(0,1)$
• For $\nu =1$, becomes Cauchy distribution

Construction: If $U\sim N(0,1)$ and $V\sim {\chi }^2(\nu )$ independently, then $T=\frac{U}{\sqrt{V/\nu }}\sim t(\nu )$

F-distribution

$X\sim F({\nu }_1,{\nu }_2)$

F-distribution with ${\nu }_1$ and ${\nu }_2$ degrees of freedom. Used to compare variances and in ANOVA.

• pdf: $\frac{\Gamma \left(\frac{{\nu }_1+{\nu }_2}{2}\right)}{\Gamma \left(\frac{{\nu }_1}{2}\right)\Gamma \left(\frac{{\nu }_2}{2}\right)}{\left(\frac{{\nu }_1}{{\nu }_2}\right)}^{\frac{{\nu }_1}{2}}\frac{x^{\frac{{\nu }_1}{2}-1}}{{\left(1+\frac{{\nu }_{1}x}{{\nu }_2}\right)}^{\frac{{\nu }_1+{\nu }_2}{2}}}$, $x>0$
• mean: $\frac{{\nu }_2}{{\nu }_2-2}$ (for ${\nu }_2>2$)
• var: $\frac{2{\nu }_2^2({\nu }_1+{\nu }_2-2)}{{\nu }_1({\nu }_2-2{)}^2({\nu }_2-4)}$ (for ${\nu }_2>4$)
• mgf: Does not exist

Key properties:

• Right-skewed distribution
• $F({\nu }_1,{\nu }_2)=\frac{1}{F({\nu }_2,{\nu }_1)}$ (reciprocal property)
• As ${\nu }_1,{\nu }_2\to \infty$, approaches normal distribution

Construction: If $U\sim {\chi }^2({\nu }_1)$ and $V\sim {\chi }^2({\nu }_2)$ independently, then $F=\frac{U/{\nu }_1}{V/{\nu }_2}\sim F({\nu }_1,{\nu }_2)$

Bivariate normal distribution

$(X,Y)\sim N_2({\mu }_1,{\mu }_2,{\sigma }_1^2,{\sigma }_2^2,\rho )$

Joint distribution of two normally distributed variables

• pdf: $\frac{1}{2\pi {\sigma }_1{\sigma }_2\sqrt{1-{\rho }^2}}\exp \left(\frac{-1}{2(1-{\rho }^2)}\left[\frac{(x-{\mu }_1{)}^2}{{\sigma }_1^2}-\frac{2\rho (x-{\mu }_1)(y-{\mu }_2)}{{\sigma }_1{\sigma }_2}+\frac{(y-{\mu }_2{)}^2}{{\sigma }_2^2}\right]\right)$
• mean: $E(X)={\mu }_1$, $E(Y)={\mu }_2$
• var: $\text{Var}(X)={\sigma }_1^2$, $\text{Var}(Y)={\sigma }_2^2$
• mgf: $\exp \left({\mu }_1{t}_1+{\mu }_2{t}_2+\frac{{\sigma }_1^2{t}_1^2+2\rho {\sigma }_1{\sigma }_2{t}_1{t}_2+{\sigma }_2^2{t}_2^2}{2}\right)$

Independence: $X$ and $Y$ are independent if and only if $\rho =0$

Conditional distributions:

• $Y|X=x\sim N\left({\mu }_2+\rho \frac{{\sigma }_2}{{\sigma }_1}(x-{\mu }_1),{\sigma }_2^2(1-{\rho }^2)\right)$
• $X|Y=y\sim N\left({\mu }_1+\rho \frac{{\sigma }_1}{{\sigma }_2}(y-{\mu }_2),{\sigma }_1^2(1-{\rho }^2)\right)$