1.10 Important Inequalities

<< 1.9 Some Special Expectations.md | 2.1 Distributions of Two Random Variables.md >>

This page will show overview of some famous inequalities involving expectations.

Theorem 1.10.1: Existence of lower order moments

Theorem

Let:

• $X$ random variable
• $m\in \mathbb{P}$
• $k\in \mathbb{P},k\le m$

If $E(X^m)$ exists, then $E(X^k)$ exists

If a higher-order moment exists, then all lower-order moments must also exist

Theorem 1.10.2: Markov’s inequality

Theorem

Let:

• $X$ random variable
• $u(X)\to \mathbb{P}$
• $E[u(X)]$ exists $\forall c\in \mathbb{P}$

Then

$$P[u(X)\ge c]\le \frac{E[u(X)]}{c}$$

Theorem 1.10.3: Chebyshev’s inequality

Theorem

Let:

• $X$ Random Variable
• ${\sigma }^2\in \mathbb{R}$ Variance of $X$
• $\mu =E(X)$ (by Existence of Lower Order Moments, ${\sigma }^2\in \mathbb{R}$ implies that $E(X)$ exists)

Then, for every $k>0$

$$P(|X-\mu |\ge k\sigma )\le \frac{1}{k^2}$$

Or equivalently,

$$P(|X-\mu |<k\sigma )\ge 1-\frac{1}{k^2}$$

Definition 1.10.1: Convex function

Definition

Let $\varphi$ : Function defined on interval $(a,b)$, where $-\infty \le a<b\le \infty$

If $ \phi[\gamma x + (1-\gamma)y] \leq \gamma\phi(x) + (1-\gamma)\phi(y),\quad\forall x,y\in(a,b), 0<\gamma<1 $$

Then we say $\varphi$ is a convex function

Note

If the inequality is strict (i.e., $<$ instead of $\le$), then we say $\varphi$ is strictly convex.

Theorem 1.10.4

If $\varphi$ is differentiable on $(a,b)$

Then

• $\varphi$ is convex $⟺{\varphi }^{\prime }(x)\le {\varphi }^{\prime }(y),\forall a<x<y<b$
• $\varphi$ is strictly convex $⟺{\varphi }^{\prime }(x)<{\varphi }^{\prime }(y),\forall a<x<y<b$

If $\varphi$ is twice differentiable on $(a,b)$

Then

• $\varphi$ is convex $⟺{\varphi }^{\prime \prime }(x)\ge 0,\forall a<x<b$
• $\varphi$ is strictly convex $⟺{\varphi }^{\prime \prime }(x)>0,\forall a<x<b$

Theorem 1.10.5: Jensen’s inequality

Theorem

Let

• $\varphi$ is convex on an open interval $I$
• $X$ : Random variable
• ${\mathcal{S}}_X$ : Support of $X$

If

• ${\mathcal{S}}_X\subseteq I$
• $E(X)$ is finite

Then $ \phi[E(X)]\leq E[\phi(X)] $$