Tweedie distributions Tweedie's contributions included pioneering work with the
Inverse Gaussian distribution. Arguably his major achievement rests with the definition of a family of
exponential dispersion models characterized by
closure under additive and reproductive
convolution as well as under
transformations of scale that are now known as the
Tweedie exponential dispersion models. As a consequence of these properties the Tweedie exponential dispersion models are characterized by a
power law relationship between the variance and the mean which leads them to become the foci of
convergence for a
central limit like effect that acts on a wide variety of random data. The range of application of the Tweedie distributions is wide and includes: •
Taylor's law, • fluctuation scaling, •
1/f noise, • genomic structure and
evolution, • regional blood flow heterogeneity, •
multifractality.
Tweedie's formula Tweedie is credited for a formula first published in Robbins (1956), which offers "a simple empirical Bayes approach to correcting selection bias". Let \mu be a latent variable we don't observe, but we know it has a certain prior distribution p(\mu) . Let x = \mu + \epsilon be an observable, where \epsilon \sim N(0, \Sigma) is a Gaussian noise variable (so p(x|\mu) = N(x| \mu, \Sigma) ) . Let \rho(x) = \int p(x|\mu) p(\mu) \text{d} \mu be the probability density of x, then the posterior mean and variance of \mu given the observed x are: E[\mu|x] = x + \Sigma \frac{\nabla \rho(x)}{\rho(x)}; \quad Var[ \mu |x] = E[\mu\mu^T|x] - E[\mu|x]E[\mu|x]^T = \Sigma\left(\frac{\nabla^2\rho(x)}{\rho(x)}- \frac{\nabla\rho(x)\nabla\rho(x)^T}{\rho(x)^2}\right)\Sigma + \Sigma The posterior higher order moments of \mu are also obtainable as algebraic expressions of \nabla\rho,\rho, \Sigma.
Proof for first part Using \nabla N(x|\mu,\Sigma) = N(x|\mu,\Sigma) \nabla \log N(x|\mu,\Sigma) = - \Sigma^{-1} (x-\mu) N(x|\mu,\Sigma), we get \Sigma \, \frac{\nabla \rho(x)}{\rho(x)} = \frac{\Sigma \int \nabla N(x|\mu,\Sigma) \, p(\mu) \text{d} \, \mu}{\int N(x|\mu,\Sigma) \, p(\mu) \, \text{d} \mu} = - \frac{\int (x-\mu) N(x|\mu,\Sigma) \, p(\mu) \, \text{d}\mu}{\int N(x|\mu,\Sigma) \, p(\mu) \, \text{d} \mu} = - \int (x-\mu) p(\mu|x) \, \text{d} \mu = - x + E[\mu|x], where we have used Bayes' theorem to write p(\mu|x) = \frac{p(x|\mu) p(\mu)}{p(x)} = \frac{N(x|\mu,\Sigma) \, p(\mu)}{ \int N(x|\mu,\Sigma) \, p(\mu) \, \text{d} \mu }. Tweedie's formula is used in
empirical Bayes method and
diffusion models. == References ==