4.1 Sampling and Statistics

<< 3.7 *Mixture Distributions | 4.2 Confidence Intervals >>

Definition 4.1.1: Random sample

Definition

Let $X_1,\dots ,X_n$ : Random samples of size $n$

If $X_1,\dots ,X_n$ are independent and identically distributed (iid)

Then we say $X_1,\dots ,X_n$ are random samples

Definition 4.1.2: Statistic

Definition

Let $X_1,\dots ,X_n$ : Random sample

If $T=T(X_1,\dots ,X_n)$, function of random samples

Then we say $T$ is a statistic of $X_1,\dots ,X_n$

Definition: Estimator

Definition

Estimator of $\theta$ is a statistic used to estimate an unknown parameter $\theta$.

Definition 4.1.3: Unbiased estimator

Definition

Let

• $X_1,\dots ,X_n$ : Random samples, with
  • $\Omega$ : Parameter space
  • pdf $f(x;\theta ),\theta \in \Omega$
• $T$ : Statistic of $X_1,\dots ,X_n$

If $E(T)=\theta,\quad \forall \theta\in \Omega$$

Then we say $T$ is an unbiased estimator of $\theta$

Definition: Likelihood function

Definition

Let

• $X_1,\dots ,X_n$ : Random sample
• $x_1,\dots ,x_n$ : Realization of $X_1,\dots ,X_n$
• $\theta$ : Parameter
• $f(x;\theta )$ : pdf of $X_1,\dots ,X_n$

If

$$\begin{array}{rl}L(\theta )& =L(\theta ;x_1,\dots ,x_n)\\ & =\prod _{i=1}^{n}f(x_i;\theta )\end{array}$$

Then we say $L(\theta )$ is the likelihood function of $X_1,\dots ,X_n$