Bernoulli distribution If
X1, ....,
Xn are independent
Bernoulli-distributed random variables with expected value
p, then the sum
T(
X) =
X1 + ... +
Xn is a sufficient statistic for
p (here 'success' corresponds to
Xi = 1 and 'failure' to
Xi = 0; so
T is the total number of successes) This is seen by considering the joint probability distribution: : \Pr\{X=x\}=\Pr\{X_1=x_1,X_2=x_2,\ldots,X_n=x_n\}. Because the observations are independent, this can be written as : p^{x_1}(1-p)^{1-x_1} p^{x_2}(1-p)^{1-x_2}\cdots p^{x_n}(1-p)^{1-x_n} and, collecting powers of
p and 1 −
p, gives : p^{\sum x_i}(1-p)^{n-\sum x_i}=p^{T(x)}(1-p)^{n-T(x)} which satisfies the factorization criterion, with
h(
x) = 1 being just a constant. Note the crucial feature: the unknown parameter
p interacts with the data
x only via the statistic
T(
x) = Σ
xi. As a concrete application, this gives a procedure for distinguishing a
fair coin from a biased coin.
Uniform distribution If
X1, ....,
Xn are independent and
uniformly distributed on the interval [0,
θ], then
T(
X) = max(
X1, ...,
Xn) is sufficient for θ — the
sample maximum is a sufficient statistic for the population maximum. To see this, consider the joint
probability density function of
X (
X1,...,
Xn). Because the observations are independent, the pdf can be written as a product of individual densities :\begin{align} f_{\theta}(x_1,\ldots,x_n) &= \frac{1}{\theta}\mathbf{1}_{\{0\leq x_1\leq\theta\}} \cdots \frac{1}{\theta}\mathbf{1}_{\{0\leq x_n\leq\theta\}} \\[5pt] &= \frac{1}{\theta^n} \mathbf{1}_{\{0\leq\min\{x_i\}\}}\mathbf{1}_{\{\max\{x_i\}\leq\theta\}} \end{align} where
1{
...} is the
indicator function. Thus the density takes form required by the Fisher–Neyman factorization theorem, where
h(
x) =
1{min{
xi}≥0}, and the rest of the expression is a function of only
θ and
T(
x) = max{
xi}. In fact, the
minimum-variance unbiased estimator (MVUE) for
θ is : \frac{n+1}{n}T(X). This is the sample maximum, scaled to correct for the
bias, and is MVUE by the
Lehmann–Scheffé theorem. Unscaled sample maximum
T(
X) is the
maximum likelihood estimator for
θ.
Uniform distribution (with two parameters) If X_1,...,X_n are independent and
uniformly distributed on the interval [\alpha, \beta] (where \alpha and \beta are unknown parameters), then T(X_1^n)=\left(\min_{1 \leq i \leq n}X_i,\max_{1 \leq i \leq n}X_i\right) is a two-dimensional sufficient statistic for (\alpha\, , \, \beta). To see this, consider the joint
probability density function of X_1^n=(X_1,\ldots,X_n). Because the observations are independent, the pdf can be written as a product of individual densities, i.e. :\begin{align} f_{X_1^n}(x_1^n) &= \prod_{i=1}^n \left({1 \over \beta-\alpha}\right) \mathbf{1}_{ \{ \alpha \leq x_i \leq \beta \} } = \left({1 \over \beta-\alpha}\right)^n \mathbf{1}_{ \{ \alpha \leq x_i \leq \beta, \, \forall \, i = 1,\ldots,n\}} \\ &= \left({1 \over \beta-\alpha}\right)^n \mathbf{1}_{ \{ \alpha \, \leq \, \min_{1 \leq i \leq n}X_i \} } \mathbf{1}_{ \{ \max_{1 \leq i \leq n}X_i \, \leq \, \beta \} }. \end{align} The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting :\begin{align} h(x_1^n)= 1, \quad g_{(\alpha, \beta)}(x_1^n)= \left({1 \over \beta-\alpha}\right)^n \mathbf{1}_{ \{ \alpha \, \leq \, \min_{1 \leq i \leq n}X_i \} } \mathbf{1}_{ \{ \max_{1 \leq i \leq n}X_i \, \leq \, \beta \} }. \end{align} Since h(x_1^n) does not depend on the parameter (\alpha, \beta) and g_{(\alpha \, , \, \beta)}(x_1^n) depends only on x_1^n through the function T(X_1^n)= \left(\min_{1 \leq i \leq n}X_i,\max_{1 \leq i \leq n}X_i\right), the Fisher–Neyman factorization theorem implies T(X_1^n) = \left(\min_{1 \leq i \leq n}X_i,\max_{1 \leq i \leq n}X_i\right) is a sufficient statistic for (\alpha\, , \, \beta).
Poisson distribution If
X1, ....,
Xn are independent and have a
Poisson distribution with parameter
λ, then the sum
T(
X) =
X1 + ... +
Xn is a sufficient statistic for
λ. To see this, consider the joint probability distribution: : \Pr(X=x)=P(X_1=x_1,X_2=x_2,\ldots,X_n=x_n). Because the observations are independent, this can be written as : {e^{-\lambda} \lambda^{x_1} \over x_1 !} \cdot {e^{-\lambda} \lambda^{x_2} \over x_2 !} \cdots {e^{-\lambda} \lambda^{x_n} \over x_n !} which may be written as : e^{-n\lambda} \lambda^{(x_1+x_2+\cdots+x_n)} \cdot {1 \over x_1 ! x_2 !\cdots x_n ! } which shows that the factorization criterion is satisfied, where
h(
x) is the reciprocal of the product of the factorials. Note the parameter λ interacts with the data only through its sum
T(
X).
Normal distribution If X_1,\ldots,X_n are independent and
normally distributed with expected value \theta (a parameter) and known finite variance \sigma^2, then :T(X_1^n)=\overline{x}=\frac1n\sum_{i=1}^nX_i is a sufficient statistic for \theta. To see this, consider the joint
probability density function of X_1^n=(X_1,\dots,X_n). Because the observations are independent, the pdf can be written as a product of individual densities, i.e. :\begin{align} f_{X_1^n}(x_1^n) & = \prod_{i=1}^n \frac{1}{\sqrt{2\pi\sigma^2}} \exp \left (-\frac{(x_i-\theta)^2}{2\sigma^2} \right ) \\ [6pt] &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left ( -\sum_{i=1}^n \frac{(x_i-\theta)^2}{2\sigma^2} \right ) \\ [6pt] & = (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left (-\sum_{i=1}^n \frac{ \left ( \left (x_i-\overline{x} \right ) - \left (\theta-\overline{x} \right ) \right )^2}{2\sigma^2} \right ) \\ [6pt] & = (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \left(\sum_{i=1}^n(x_i-\overline{x})^2 + \sum_{i=1}^n(\theta-\overline{x})^2 -2\sum_{i=1}^n(x_i-\overline{x})(\theta-\overline{x})\right) \right) \\ [6pt] &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \left (\sum_{i=1}^n(x_i-\overline{x})^2 + n(\theta-\overline{x})^2 \right ) \right ) && \sum_{i=1}^n(x_i-\overline{x})(\theta-\overline{x})=0 \\ [6pt] &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \sum_{i=1}^n (x_i-\overline{x})^2 \right ) \exp \left (-\frac{n}{2\sigma^2} (\theta-\overline{x})^2 \right ) \end{align} The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting :\begin{align} h(x_1^n) &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \sum_{i=1}^n (x_i-\overline{x})^2 \right ) \\[6pt] g_\theta(x_1^n) &= \exp \left (-\frac{n}{2\sigma^2} (\theta-\overline{x})^2 \right ) \end{align} Since h(x_1^n) does not depend on the parameter \theta and g_{\theta}(x_1^n) depends only on x_1^n through the function :T(X_1^n)=\overline{x}=\frac1n\sum_{i=1}^nX_i, the Fisher–Neyman factorization theorem implies T(X_1^n) is a sufficient statistic for \theta. If \sigma^2 is unknown and since s^2 = \frac{1}{n-1} \sum_{i=1}^n \left(x_i - \overline{x} \right)^2 , the above likelihood can be rewritten as :\begin{align} f_{X_1^n}(x_1^n)= (2\pi\sigma^2)^{-n/2} \exp \left( -\frac{n-1}{2\sigma^2}s^2 \right) \exp \left (-\frac{n}{2\sigma^2} (\theta-\overline{x})^2 \right ) . \end{align} The Fisher–Neyman factorization theorem still holds and implies that (\overline{x},s^2) is a joint sufficient statistic for ( \theta , \sigma^2) .
Exponential distribution If X_1,\dots,X_n are independent and
exponentially distributed with expected value
θ (an unknown real-valued positive parameter), then T(X_1^n)=\sum_{i=1}^nX_i is a sufficient statistic for θ. To see this, consider the joint
probability density function of X_1^n=(X_1,\dots,X_n). Because the observations are independent, the pdf can be written as a product of individual densities, i.e. :\begin{align} f_{X_1^n}(x_1^n) &= \prod_{i=1}^n {1 \over \theta} \, e^{ {-1 \over \theta}x_i } = {1 \over \theta^n}\, e^{ {-1 \over \theta} \sum_{i=1}^nx_i }. \end{align} The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting :\begin{align} h(x_1^n)= 1,\,\,\, g_{\theta}(x_1^n)= {1 \over \theta^n}\, e^{ {-1 \over \theta} \sum_{i=1}^nx_i }. \end{align} Since h(x_1^n) does not depend on the parameter \theta and g_{\theta}(x_1^n) depends only on x_1^n through the function T(X_1^n)=\sum_{i=1}^nX_i the Fisher–Neyman factorization theorem implies T(X_1^n)=\sum_{i=1}^nX_i is a sufficient statistic for \theta.
Gamma distribution If X_1,\dots,X_n are independent and distributed as a
\Gamma(\alpha \, , \, \beta), where \alpha and \beta are unknown parameters of a
Gamma distribution, then T(X_1^n) = \left( \prod_{i=1}^n{X_i} , \sum_{i=1}^n X_i \right) is a two-dimensional sufficient statistic for (\alpha, \beta). To see this, consider the joint
probability density function of X_1^n=(X_1,\dots,X_n). Because the observations are independent, the pdf can be written as a product of individual densities, i.e. :\begin{align} f_{X_1^n}(x_1^n) &= \prod_{i=1}^n \left({1 \over \Gamma(\alpha) \beta^\alpha}\right) x_i^{\alpha -1} e^{(-1/\beta)x_i} \\[5pt] &= \left({1 \over \Gamma(\alpha) \beta^\alpha}\right)^n \left(\prod_{i=1}^n x_i\right)^{\alpha-1} e^{{-1 \over \beta} \sum_{i=1}^n x_i}. \end{align} The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting :\begin{align} h(x_1^n)= 1,\,\,\, g_{(\alpha \, , \, \beta)}(x_1^n)= \left({1 \over \Gamma(\alpha) \beta^{\alpha}}\right)^n \left(\prod_{i=1}^n x_i\right)^{\alpha-1} e^{{-1 \over \beta} \sum_{i=1}^n x_i}. \end{align} Since h(x_1^n) does not depend on the parameter (\alpha\, , \, \beta) and g_{(\alpha \, , \, \beta)}(x_1^n) depends only on x_1^n through the function T(x_1^n)= \left( \prod_{i=1}^n x_i, \sum_{i=1}^n x_i \right), the Fisher–Neyman factorization theorem implies T(X_1^n)= \left( \prod_{i=1}^n X_i, \sum_{i=1}^n X_i \right) is a sufficient statistic for (\alpha\, , \, \beta). ==Rao–Blackwell theorem==