Unique MVUE (UMVUE)

Theorem

Let

• $X_1,\dots ,X_n$ : Random sample, with
  • pdf/pmf $f(x;\theta )$, $\theta \in \Omega$
• $Y_1=u_1(X_1,\dots ,X_n)$ : Complete sufficient statistic for $\theta$

If $\phi (Y_1)$ is an unbiased estimator of $\theta$

Then $\phi (Y_1)$ is the unique MVUE (UMVUE) of $\theta$

Proof

By Rao-Blackwell theorem, if $Y_2$ is any unbiased estimate of $\theta$, then $E[Y_2|Y_1]$ is an unbiased estimate of $\theta$ with $\text{Var}([E(Y_2|Y_1)])\le \text{Var}(Y_2)$.

But $E[Y_2|Y_1]$ is a function of $Y_1$, so by completeness it must concide with $\phi (Y_1)$.

Thus regardless of the particular value of $\theta$, ${\text{Var}}_{\theta }[\phi (Y_1)]\le {\text{Var}}_{\theta }(Y_2)$.

Remark

In the reference book, this theorem is called Lehmann and Scheffe theorem.

The Lehmann-Scheffé theorem states that if $T$ is a complete sufficient statistic for a parameter $\theta$, then a function of it is UMVUE, if it’s unbiased and exists

Example

Let $X_1,X_2,\dots ,X_n$ represent a random sample from the discrete distribution with pdf $f(x;\theta )={\theta }^x(1-\theta {)}^{1-x},x=0,1,0<\theta <1$, zero elsewhere.

Show that $Y_1={\sum }_{i=1}^n{X}_i$ is a complete, sufficient statistic for $\theta$. Find the unique function of $Y_1$ that is the unbiased minimum variance estimator for $\theta$

$$\begin{array}{rl}{\theta }^x(1-\theta {)}^{1-x}& =\exp [x\ln \theta +(1-x)\ln (1-\theta )]\\ & =\exp [x\ln \theta +\ln (1-\theta )-x\ln (1-\theta )]\end{array}$$

Then $f(x;\theta )$ has the form of regular exponential class, with

• $\mathcal{S}=\{0,1\}$ : Independent of $\theta$
• $p(\theta )=\ln \theta$ : Continuous, nontrivial function
• $K(x)=x$ : Nontrivial function
• $H(x)=\ln (1-\theta )$
• $q(\theta )=x\ln (1-\theta )$

By Neyman Theorem, $Y_1$ is a sufficient statistic:

$$\begin{array}{rl}L(\theta )& =\prod _{i=1}^n{\theta }^x(1-\theta {)}^{1-x_i}\\ & ={\theta }^{nx}(1-\theta {)}^{n-{\sum }_{i=1}^n{x}_i}\end{array}$$

Notice that $X\sim Bernoulli(\theta )$, thus $Y_1={\sum }_{i=1}^n{X}_i\sim Binom(n,\theta )$, and $E(Y_1)=n\theta$.

So, $Y_2=\frac{Y_1}{n}$ is an unbiased estimator for $\theta$, and becuase $Y_1$ is complete sufficient statistic, so is $Y_2$.

As a result, by definition of Unique MVUE (UMVUE), $Y_2$ is an unbiased minimum variance estimator for $\theta$ .

Let $X_1,X_2,\dots ,X_n$ random sample of size $n$ from distribution of $Gamma(3,\beta )$, with $\beta >0$

$$\begin{array}{rl}\frac{1}{\Gamma (3){\beta }^{\alpha }}{x}^{\alpha -1}{e}^{-x/\beta }& =\frac{1}{2{\beta }^3}{x}^2{e}^{-x/\beta }\\ & =\exp \left[-\ln 2+3\ln \beta +2\ln x-\frac{x}{\beta }\right]\end{array}$$

• $p(\beta )=-\frac{1}{\beta }$
• $K(x)=x$
• $H(x)=2\ln x$
• $q(\theta )=-3\ln \beta$

$$\begin{array}{rl}\prod _{i=1}^n\frac{1}{2{\beta }^3}{x}^2{e}^{-x_i/\beta }& =\frac{1}{2^n{\beta }^{3n}}{x}^{2n}{e}^{-1/\beta {\sum }_{i=1}^n{x}_i}\end{array}$$

Since $E(X)=3\beta$, we have $E\left(\frac{X}{3}\right)=\beta$.

Let

$$Y_2=\frac{1}{3}\bar{X}=\frac{1}{3n}\sum _{i=1}^n{X}_i$$

This statistic $Y_2$ is unbiased for $\beta$

$$\begin{array}{rl}E(Y_2)& =E\left(\frac{1}{3}\bar{X}\right)\\ & =\frac{1}{3}E(\bar{X})\\ & =\frac{1}{3}E(X)\\ & =\frac{1}{3}\cdot 3\beta \\ & =\beta \end{array}$$