Minimum-variance unbiased estimator

Consider estimation of g(\theta) based on data X_1, X_2, \ldots, X_n i.i.d. from some member of a family of densities p_\theta, \theta \in \Omega, where \Omega is the parameter space. An estimator \delta(X_1, X_2, \ldots, X_n) of g(\theta) is unbiased if \operatorname{E}(\delta(X_1, X_2, \ldots, X_n))=\operatorname{E}(g(\theta)). Furthermore, an unbiased estimator is UMVUE if \forall \theta \in \Omega, \operatorname{var}(\delta(X_1, X_2, \ldots, X_n)) \leq \operatorname{var}(\tilde{\delta}(X_1, X_2, \ldots, X_n)) for any other unbiased estimator \tilde{\delta}. If an unbiased estimator of g(\theta) exists, then one can prove there is an essentially unique MVUE. Using the Rao–Blackwell theorem one can also prove that determining the MVUE is simply a matter of finding a complete sufficient statistic for the family p_\theta, \theta \in \Omega and conditioning any unbiased estimator on it. Further, by the Lehmann–Scheffé theorem, an unbiased estimator that is a function of a complete, sufficient statistic is the UMVUE. Put formally, suppose \delta(X_1, X_2, \ldots, X_n) is unbiased for g(\theta), and that T is a complete sufficient statistic for the family of densities. Then \eta(X_1, X_2, \ldots, X_n) = \operatorname{E}(\delta(X_1, X_2, \ldots, X_n)\mid T)\, is the MVUE for g(\theta). A Bayesian analog is a Bayes estimator, particularly with minimum mean square error (MMSE). ==Estimator selection==

An efficient estimator need not exist, but if it does and if it is unbiased, it is the MVUE. Since the mean squared error (MSE) of an estimator δ is \operatorname{MSE}(\delta) = \operatorname{var}(\delta) +[ \operatorname{bias}(\delta)]^2 \ the MVUE minimizes MSE among unbiased estimators. In some cases biased estimators have lower MSE because they have a smaller variance than does any unbiased estimator; see estimator bias. ==Example==

Consider the data to be a single observation from an absolutely continuous distribution on \mathbb{R} with density p_\theta(x) = \frac{ \theta e^{-x} }{\left(1 + e^{-x}\right)^{\theta + 1} }, where θ > 0, and we wish to find the UMVU estimator of g(\theta) = \frac 1 {\theta^2} First we recognize that the density can be written as \frac{ e^{-x} } { 1 + e^{-x} } \exp\left( -\theta \log(1 + e^{-x}) + \log(\theta)\right) which is an exponential family with sufficient statistic T = \log(1 + e^{-X}). In fact this is a full rank exponential family, and therefore T is complete sufficient. See exponential family for a derivation which shows \operatorname{E}(T) = \frac 1 \theta,\quad \operatorname{var}(T) = \frac 1 {\theta^2} Therefore, \operatorname{E}(T^2) = \frac 2 {\theta^2} Here we use Lehmann–Scheffé theorem to get the MVUE. Clearly, \delta(X) = T^2/2 is unbiased and T = \log(1 + e^{-X}) is complete sufficient, thus the UMVU estimator is \eta(X) = \operatorname{E}(\delta(X) \mid T) = \operatorname{E} \left( \left. \frac{T^2} 2 \,\right|\, T \right) = \frac{T^2} 2 = \frac{\log\left(1 + e^{-X}\right)^2}{2} This example illustrates that an unbiased function of the complete sufficient statistic will be UMVU, as Lehmann–Scheffé theorem states. == Other examples ==

• For a normal distribution with unknown mean and variance, the sample mean and (unbiased) sample variance are the MVUEs for the population mean and population variance. • :However, the sample standard deviation is not unbiased for the population standard deviation – see unbiased estimation of standard deviation. • :Further, for other distributions the sample mean and sample variance are not in general MVUEs – for a uniform distribution with unknown upper and lower bounds, the mid-range is the MVUE for the population mean. • If k exemplars are chosen (without replacement) from a discrete uniform distribution over the set {1, 2, ..., N} with unknown upper bound N, the MVUE for N is: \frac{k+1}{k} m - 1, where m is the sample maximum. This is a scaled and shifted (so unbiased) transform of the sample maximum, which is a sufficient and complete statistic. See German tank problem for details. == See also ==