Sufficient Statistic

Definition

Let

• $X_1,\dots ,X_n$ : Random sample, with joint pdf/pmf $f(x;\theta )$, $\theta \in \Omega$
• $Y=u(\mathbf{X})$ : Statistic, with pdf/pmf $f_Y(y;\theta )$

Then $Y$ is a sufficient statistic for $\theta$

If and only if

• $\frac{{\prod }_{i=1}^{n}f(x_i;\theta )}{f_Y[u(\mathbf{x});\theta ]}=H(\mathbf{x})$
• $H(\mathbf{X})$ does not depend upon $\theta$

Tip

A sufficient statistic $Y$ for $\theta$ (function of $\mathbf{x}$) completely explains $\theta$ for any $\mathbf{x}$ such that $Y=y$, meaning:

• For any $\mathbf{x}$ such that $Y=y$, $\mathbf{x}$ no longer tells anything more about $\theta$ beyond what $Y=y$ already explains

Remark

To prove $Y$ is a sufficient statistic, we must show that $\frac{f}{f_Y}$ does not depend on $\theta$, i.e., does not have any $\theta$ terms in it.

A sufficient statistic captures all the information about $\theta$ contained in the sample, so the ratio of joint to marginal densities should be free of $\theta$.

Also, a sufficient statistic does not require the random variables to be independent.

Subdefinition

• Complete Sufficient Statistic
• Jointly Sufficient Statistic

Related theorems

• Neyman Theorem