Likelihood function

The likelihood function, parameterized by a (possibly multivariate) parameter \theta, is usually defined differently for discrete and continuous probability distributions (a more general definition is discussed below). Given a probability density or mass function x\mapsto f(x \mid \theta), where x is a realization of the random variable X, the likelihood function is \theta\mapsto f(x \mid \theta), often written \mathcal{L}(\theta \mid x). In other words, when f(x\mid\theta) is viewed as a function of x with \theta fixed, it is a probability density function, and when viewed as a function of \theta with x fixed, it is a likelihood function. In the frequentist paradigm, the notation f(x\mid\theta) is often avoided and instead f(x;\theta) or f(x,\theta) are used to indicate that \theta is regarded as a fixed unknown quantity rather than as a random variable being conditioned on. The likelihood function does not specify the probability that \theta is the truth, given the observed sample X = x. Such an interpretation is a common error, with potentially disastrous consequences (see prosecutor's fallacy). Discrete probability distribution Let X be a discrete random variable with probability mass function p depending on a parameter \theta. Then the function \mathcal{L}(\theta \mid x) = p_\theta (x) = P_\theta (X=x) = \text{Pr}\{ X=x \mid \Theta=\theta \} , considered as a function of \theta, a possible value of the deterministic but unknown parameter \Theta, is the likelihood function, given the outcome x of the random variable X. Sometimes the probability of "the value x of X for the parameter value \theta" is written as or . The likelihood is the probability that a particular outcome x is observed when the true value of the parameter is \theta, equivalent to the probability mass on x; it is not a probability density over the parameter \theta. The likelihood, \mathcal{L}(\theta \mid x) , should not be confused with P(\theta \mid x), which is the posterior probability of \theta given the data x. Example Consider a simple statistical model of a coin flip: a single parameter p_\text{H} that expresses the "fairness" of the coin. The parameter is the probability that a coin lands heads up ("H") when tossed. p_\text{H} can take on any value within the range 0.0 to 1.0. For a perfectly fair coin, p_\text{H} = 0.5. Imagine flipping a fair coin twice, and observing two heads in two tosses ("HH"). Assuming that each successive coin flip is i.i.d., then the probability of observing HH is P(\text{HH} \mid p_\text{H}=0.5) = 0.5^2 = 0.25. Equivalently, the likelihood of observing "HH" assuming p_\text{H} = 0.5 is \mathcal{L}(p_\text{H}=0.5 \mid \text{HH}) = 0.25. This is not the same as saying that P(p_\text{H} = 0.5 \mid \text{HH}) = 0.25, a conclusion which could only be reached via Bayes' theorem given knowledge about the marginal probabilities P(p_\text{H} = 0.5) and P(\text{HH}). Now suppose that the coin is not a fair coin, but instead that p_\text{H} = 0.3. Then the probability of two heads on two flips is P(\text{HH} \mid p_\text{H}=0.3) = 0.3^2 = 0.09. Hence \mathcal{L}(p_\text{H}=0.3 \mid \text{HH}) = 0.09. More generally, for each value of p_\text{H}, we can calculate the corresponding likelihood. The result of such calculations is displayed in Figure 1. The integral of \mathcal{L} over [0, 1] is 1/3; likelihoods need not integrate or sum to one over the parameter space. Continuous probability distribution Let X be a random variable following an absolutely continuous probability distribution with density function f (a function of x) which depends on a parameter \theta. Then the function \mathcal{L}(\theta \mid x) = f_\theta (x), considered as a function of \theta, is the likelihood function (of \theta, given the outcome X=x). Again, \mathcal{L} is not a probability density or mass function over \theta, despite being a function of \theta given the observation X = x. Relationship between the likelihood and probability density functions The use of the probability density in specifying the likelihood function above is justified as follows. Given an observation x_j, the likelihood for the interval [x_j, x_j + h], where h > 0 is a constant, is given by \mathcal{L}(\theta \mid x \in [x_j, x_j{+}h]) . Observe that \mathop\operatorname{arg\,max}_\theta \mathcal{L}(\theta\mid x \in [x_j, x_j{+}h]) = \mathop\operatorname{arg\,max}_\theta \frac{1}{h} \mathcal{L}(\theta\mid x \in [x_j, x_j{+}h]) , since h is positive and constant. Because \begin{align} \mathop\operatorname{arg\,max}_\theta \frac 1 h \mathcal{L}(\theta\mid x \in [x_j, x_j{+}h]) &= \mathop\operatorname{arg\,max}_\theta \frac 1 h \Pr(x_j \leq x \leq x_j{+}h \mid \theta) \\ &= \mathop\operatorname{arg\,max}_\theta \frac 1 h \int_{x_j}^{x_j+h} f(x\mid \theta) \,dx, \end{align} where f(x\mid \theta) is the probability density function, it follows that \mathop\operatorname{arg\,max}_\theta \mathcal{L}(\theta\mid x \in [x_j, x_j + h]) = \mathop\operatorname{arg\,max}_\theta \frac{1}{h} \int_{x_j}^{x_j+h} f(x\mid\theta) \,dx . The first fundamental theorem of calculus provides that \lim_{h \to 0^{+}} \frac 1 h \int_{x_j}^{x_j+h} f(x\mid\theta) \,dx = f(x_j \mid \theta). Then \begin{align} \mathop\operatorname{arg\,max}_\theta \mathcal{L}(\theta\mid x_j) &= \mathop\operatorname{arg\,max}_\theta \left[ \lim_{h\to 0^+} \mathcal{L}(\theta\mid x \in [x_j,\, x_j{+}h]) \right] \\[4pt] &= \mathop\operatorname{arg\,max}_\theta \left[ \lim_{h\to 0^{+}} \frac{1}{h} \int_{x_j}^{x_j+h} f(x\mid\theta) \,dx \right] \\[4pt] &= \mathop\operatorname{arg\,max}_\theta f(x_j \mid \theta). \end{align} Therefore, \mathop\operatorname{arg\,max}_\theta \mathcal{L}(\theta\mid x_j) = \mathop\operatorname{arg\,max}_\theta f(x_j \mid \theta), and so maximizing the probability density at x_j amounts to maximizing the likelihood of the specific observation x_j . In general In measure-theoretic probability theory, the density function is defined as the Radon–Nikodym derivative of the probability distribution relative to a common dominating measure. The likelihood function is this density interpreted as a function of the parameter, rather than the random variable. Thus, we can construct a likelihood function for any distribution, whether discrete, continuous, a mixture, or otherwise. (Likelihoods are comparable, e.g. for parameter estimation, only if they are Radon–Nikodym derivatives with respect to the same dominating measure.) The above discussion of the likelihood for discrete random variables uses the counting measure, under which the probability density at any outcome equals the probability of that outcome. Likelihoods for mixed continuous–discrete distributions The above can be extended in a simple way to allow consideration of distributions which contain both discrete and continuous components. Suppose that the distribution consists of a number of discrete probability masses p_k (\theta) and a density f(x\mid\theta), where the sum of all the p's added to the integral of f is always one. Assuming that it is possible to distinguish an observation corresponding to one of the discrete probability masses from one which corresponds to the density component, the likelihood function for an observation from the continuous component can be dealt with in the manner shown above. For an observation from the discrete component, the likelihood function for an observation from the discrete component is simply \mathcal{L}(\theta \mid x )= p_k(\theta), where k is the index of the discrete probability mass corresponding to observation x, because maximizing the probability mass (or probability) at x amounts to maximizing the likelihood of the specific observation. The fact that the likelihood function can be defined in a way that includes contributions that are not commensurate (the density and the probability mass) arises from the way in which the likelihood function is defined up to a constant of proportionality, where this "constant" can change with the observation x, but not with the parameter \theta. Regularity conditions In the context of parameter estimation, the likelihood function is usually assumed to obey certain conditions, known as regularity conditions. These conditions are in various proofs involving likelihood functions, and need to be verified in each particular application. For maximum likelihood estimation, the existence of a global maximum of the likelihood function is of the utmost importance. By the extreme value theorem, it suffices that the likelihood function is continuous on a compact parameter space for the maximum likelihood estimator to exist. While the continuity assumption is usually met, the compactness assumption about the parameter space is often not, as the bounds of the true parameter values might be unknown. In that case, concavity of the likelihood function plays a key role. More specifically, if the likelihood function is twice continuously differentiable on the k-dimensional parameter space \Theta assumed to be an open connected subset of \mathbb{R}^{k} \,, there exists a unique maximum \hat{\theta} \in \Theta if the matrix of second partials \mathbf{H}(\theta) \equiv \left[\, \frac{ \partial^2 L }{ \partial \theta_i \, \partial \theta_j } \,\right]_{i,j=1,1}^{n_\mathrm{i},n_\mathrm{j}} \; is negative definite for every \theta \in \Theta at which the gradient \nabla L \equiv \left[ \frac{ \partial L }{\partial \theta_i} \right]_{i=1}^{n_\mathrm{i}} vanishes, and if the likelihood function approaches a constant on the boundary of the parameter space, \partial \Theta, i.e., \lim_{\theta \to \partial \Theta} L(\theta) = 0 \;, which may include the points at infinity if \Theta is unbounded. Mäkeläinen and co-authors prove this result using Morse theory while informally appealing to a mountain pass property. Mascarenhas restates their proof using the mountain pass theorem. In the proofs of consistency and asymptotic normality of the maximum likelihood estimator, additional assumptions are made about the probability densities that form the basis of a particular likelihood function. These conditions were first established by Chanda. In particular, for almost all x, and for all \, \theta \in \Theta \,, \frac{\partial \log f}{\partial \theta_r} \,, \quad \frac{\partial^2 \log f}{\partial \theta_r \partial \theta_s} \,, \quad \frac{\partial^3 \log f}{\partial \theta_r \, \partial \theta_s \, \partial \theta_t} \, exist for all \, r, s, t = 1, 2, \ldots, k \, in order to ensure the existence of a Taylor expansion. Second, for almost all x and for every \, \theta \in \Theta \, it must be that \left| \frac{\partial f}{\partial \theta_r} \right| where H is such that \, \int_{-\infty}^{\infty} H_{rst}(z) \, dz \leq M This boundedness of the derivatives is needed to allow for differentiation under the integral sign. And lastly, it is assumed that the information matrix, \mathbf{I}(\theta) = \int_{-\infty}^{\infty} \frac{\partial \log f}{\partial \theta_r}\ \frac{\partial \log f}{\partial \theta_s}\ f\, dz is positive definite and \, \left| \mathbf{I}(\theta) \right| \, is finite. This ensures that the score has a finite variance. The above conditions are sufficient, but not necessary. That is, a model that does not meet these regularity conditions may or may not have a maximum likelihood estimator of the properties mentioned above. Further, in case of non-independently or non-identically distributed observations additional properties may need to be assumed. In Bayesian statistics, almost identical regularity conditions are imposed on the likelihood function in order to proof asymptotic normality of the posterior probability, and therefore to justify a Laplace approximation of the posterior in large samples. ==Likelihood ratio and relative likelihood==

Likelihood ratio A likelihood ratio is the ratio of any two specified likelihoods, frequently written as: \Lambda(\theta_1:\theta_2 \mid x) = \frac{\mathcal{L}(\theta_1 \mid x)}{\mathcal{L}(\theta_2 \mid x)}. The likelihood ratio is central to likelihoodist statistics: the law of likelihood states that the degree to which data (considered as evidence) supports one parameter value versus another is measured by the likelihood ratio. In frequentist inference, the likelihood ratio is the basis for a test statistic, the so-called likelihood-ratio test. By the Neyman–Pearson lemma, this is the most powerful test for comparing two simple hypotheses at a given significance level. Numerous other tests can be viewed as likelihood-ratio tests or approximations thereof. The asymptotic distribution of the log-likelihood ratio, considered as a test statistic, is given by Wilks' theorem. The likelihood ratio is also of central importance in Bayesian inference, where it is known as the Bayes factor, and is used in Bayes' rule. Stated in terms of odds, Bayes' rule states that the posterior odds of two alternatives, and , given an event , is the prior odds, times the likelihood ratio. As an equation: O(A_1:A_2 \mid B) = O(A_1:A_2) \cdot \Lambda(A_1:A_2 \mid B). The likelihood ratio is not directly used in AIC-based statistics. Instead, what is used is the relative likelihood of models (see below). In evidence-based medicine, likelihood ratios are used in diagnostic testing to assess the value of performing a diagnostic test. Relative likelihood function Since the actual value of the likelihood function depends on the sample, it is often convenient to work with a standardized measure. Suppose that the maximum likelihood estimate for the parameter is \hat{\theta}. Relative plausibilities of other values may be found by comparing the likelihoods of those other values with the likelihood of \hat{\theta}. The relative likelihood of is defined to be R(\theta) = \frac{\mathcal{L}(\theta \mid x)}{\mathcal{L}(\hat{\theta} \mid x)}. Thus, the relative likelihood is the likelihood ratio (discussed above) with the fixed denominator \mathcal{L}(\hat{\theta}). This corresponds to standardizing the likelihood to have a maximum of 1. Likelihood region A likelihood region is the set of all values of whose relative likelihood is greater than or equal to a given threshold. In terms of percentages, a % likelihood region for is defined to be \left\{\theta : R(\theta) \ge \frac p {100} \right\}. If is a single real parameter, a % likelihood region will usually comprise an interval of real values. If the region does comprise an interval, then it is called a likelihood interval. Likelihood intervals, and more generally likelihood regions, are used for interval estimation within likelihoodist statistics: they are similar to confidence intervals in frequentist statistics and credible intervals in Bayesian statistics. Likelihood intervals are interpreted directly in terms of relative likelihood, not in terms of coverage probability (frequentism) or posterior probability (Bayesianism). Given a model, likelihood intervals can be compared to confidence intervals. If is a single real parameter, then under certain conditions, a 14.65% likelihood interval (about 1:7 likelihood) for will be the same as a 95% confidence interval (19/20 coverage probability). In a slightly different formulation suited to the use of log-likelihoods (see Wilks' theorem), the test statistic is twice the difference in log-likelihoods and the probability distribution of the test statistic is approximately a chi-squared distribution with degrees-of-freedom (df) equal to the difference in df's between the two models (therefore, the −2 likelihood interval is the same as the 0.954 confidence interval; assuming difference in df's to be 1). ==Likelihoods that eliminate nuisance parameters==

In many cases, the likelihood is a function of more than one parameter but interest focuses on the estimation of only one, or at most a few of them, with the others being considered as nuisance parameters. Several alternative approaches have been developed to eliminate such nuisance parameters, so that a likelihood can be written as a function of only the parameter (or parameters) of interest: the main approaches are profile, conditional, and marginal likelihoods. These approaches are also useful when a high-dimensional likelihood surface needs to be reduced to one or two parameters of interest in order to allow a graph. Profile likelihood It is possible to reduce the dimensions by concentrating the likelihood function for a subset of parameters by expressing the nuisance parameters as functions of the parameters of interest and replacing them in the likelihood function. In general, for a likelihood function depending on the parameter vector \mathbf{\theta} that can be partitioned into \mathbf{\theta} = \left( \mathbf{\theta}_{1} : \mathbf{\theta}_{2} \right), and where a correspondence \mathbf{\hat{\theta}}_{2} = \mathbf{\hat{\theta}}_{2} \left( \mathbf{\theta}_{1} \right) can be determined explicitly, concentration reduces computational burden of the original maximization problem. For instance, in a linear regression with normally distributed errors, \mathbf{y} = \mathbf{X} \beta + u, the coefficient vector could be partitioned into \beta = \left[ \beta_{1} : \beta_{2} \right] (and consequently the design matrix \mathbf{X} = \left[ \mathbf{X}_{1} : \mathbf{X}_{2} \right]). Maximizing with respect to \beta_{2} yields an optimal value function \beta_{2} (\beta_{1}) = \left( \mathbf{X}_{2}^{\mathsf{T}} \mathbf{X}_{2} \right)^{-1} \mathbf{X}_{2}^{\mathsf{T}} \left( \mathbf{y} - \mathbf{X}_{1} \beta_{1} \right). Using this result, the maximum likelihood estimator for \beta_{1} can then be derived as \hat{\beta}_{1} = \left( \mathbf{X}_{1}^{\mathsf{T}} \left( \mathbf{I} - \mathbf{P}_{2} \right) \mathbf{X}_{1} \right)^{-1} \mathbf{X}_{1}^{\mathsf{T}} \left( \mathbf{I} - \mathbf{P}_{2} \right) \mathbf{y} where \mathbf{P}_{2} = \mathbf{X}_{2} \left( \mathbf{X}_{2}^{\mathsf{T}} \mathbf{X}_{2} \right)^{-1} \mathbf{X}_{2}^{\mathsf{T}} is the projection matrix of \mathbf{X}_{2}. This result is known as the Frisch–Waugh–Lovell theorem. Since graphically the procedure of concentration is equivalent to slicing the likelihood surface along the ridge of values of the nuisance parameter \beta_{2} that maximizes the likelihood function, creating an isometric profile of the likelihood function for a given \beta_{1}, the result of this procedure is also known as profile likelihood. In addition to being graphed, the profile likelihood can also be used to compute confidence intervals that often have better small-sample properties than those based on asymptotic standard errors calculated from the full likelihood. Conditional likelihood Sometimes it is possible to find a sufficient statistic for the nuisance parameters, and conditioning on this statistic results in a likelihood which does not depend on the nuisance parameters. One example occurs in 2×2 tables, where conditioning on all four marginal totals leads to a conditional likelihood based on the non-central hypergeometric distribution. This form of conditioning is also the basis for Fisher's exact test. Marginal likelihood Sometimes we can remove the nuisance parameters by considering a likelihood based on only part of the information in the data, for example by using the set of ranks rather than the numerical values. Another example occurs in linear mixed models, where considering a likelihood for the residuals only after fitting the fixed effects leads to residual maximum likelihood estimation of the variance components. Partial likelihood A partial likelihood is an adaption of the full likelihood such that only a part of the parameters (the parameters of interest) occur in it. {{citation |last=Cox |first=D. R. |author-link=David Cox (statistician) |title=Partial likelihood |journal=Biometrika |year=1975 |volume=62 |issue=2 |pages=269–276 |doi=10.1093/biomet/62.2.269 |mr=0400509 ==Products of likelihoods==

The likelihood, given two or more independent events, is the product of the likelihoods of each of the individual events: \Lambda(A \mid X_1 \land X_2) = \Lambda(A \mid X_1) \cdot \Lambda(A \mid X_2). This follows from the definition of independence in probability: the probabilities of two independent events happening, given a model, is the product of the probabilities. This is particularly important when the events are from independent and identically distributed random variables, such as independent observations or sampling with replacement. In such a situation, the likelihood function factors into a product of individual likelihood functions. The empty product has value 1, which corresponds to the likelihood, given no event, being 1: before any data, the likelihood is always 1. This is similar to a uniform prior in Bayesian statistics, but in likelihoodist statistics this is not an improper prior because likelihoods are not integrated. ==Log-likelihood==

Log-likelihood function is the logarithm of the likelihood function, often denoted by a lowercase or , to contrast with the uppercase or \mathcal{L} for the likelihood. Because logarithms are strictly increasing functions, maximizing the likelihood is equivalent to maximizing the log-likelihood. But for practical purposes it is more convenient to work with the log-likelihood function in maximum likelihood estimation, in particular since most common probability distributions—notably the exponential family—are only logarithmically concave, and concavity of the objective function plays a key role in the maximization. Given the independence of each event, the overall log-likelihood of intersection equals the sum of the log-likelihoods of the individual events. This is analogous to the fact that the overall log-probability is the sum of the log-probability of the individual events. In addition to the mathematical convenience from this, the adding process of log-likelihood has an intuitive interpretation, as often expressed as "support" from the data. When the parameters are estimated using the log-likelihood for the maximum likelihood estimation, each data point is used by being added to the total log-likelihood. As the data can be viewed as an evidence that support the estimated parameters, this process can be interpreted as "support from independent evidence adds", and the log-likelihood is the "weight of evidence". Interpreting negative log-probability as information content or surprisal, the support (log-likelihood) of a model, given an event, is the negative of the surprisal of the event, given the model: a model is supported by an event to the extent that the event is unsurprising, given the model. A logarithm of a likelihood ratio is equal to the difference of the log-likelihoods: \log \frac{\mathcal{L}(A)}{\mathcal{L}(B)} = \log \mathcal{L}(A) - \log \mathcal{L}(B) = \ell(A) - \ell(B). Just as the likelihood, given no event, being 1, the log-likelihood, given no event, is 0, which corresponds to the value of the empty sum: without any data, there is no support for any models. Graph The graph of the log-likelihood is called the support curve (in the univariate case). In the multivariate case, the concept generalizes into a support surface over the parameter space. It has a relation to, but is distinct from, the support of a distribution. The term was coined by A. W. F. Edwards A similar result can be established using Rolle's theorem. The second derivative evaluated at \hat{\theta}, known as Fisher information, determines the curvature of the likelihood surface, and thus indicates the precision of the estimate. Exponential families The log-likelihood is also particularly useful for exponential families of distributions, which include many of the common parametric probability distributions. The probability distribution function (and thus likelihood function) for exponential families contain products of factors involving exponentiation. The logarithm of such a function is a sum of products, again easier to differentiate than the original function. An exponential family is one whose probability density function is of the form (for some functions, writing \langle -, - \rangle for the inner product): p(x \mid \boldsymbol \theta) = h(x) \exp\Big(\langle \boldsymbol\eta({\boldsymbol \theta}), \mathbf{T}(x)\rangle -A({\boldsymbol \theta}) \Big). Each of these terms has an interpretation, but simply switching from probability to likelihood and taking logarithms yields the sum: \ell(\boldsymbol \theta \mid x) = \langle \boldsymbol\eta({\boldsymbol \theta}), \mathbf{T}(x)\rangle - A({\boldsymbol \theta}) + \log h(x). The \boldsymbol \eta(\boldsymbol \theta) and h(x) each correspond to a change of coordinates, so in these coordinates, the log-likelihood of an exponential family is given by the simple formula: \ell(\boldsymbol \eta \mid x) = \langle \boldsymbol\eta, \mathbf{T}(x)\rangle - A({\boldsymbol \eta}). In words, the log-likelihood of an exponential family is inner product of the natural parameter and the sufficient statistic {{tmath|\mathbf{T}(x)}}, minus the normalization factor (log-partition function) {{tmath|A({\boldsymbol \eta})}}. Thus for example the maximum likelihood estimate can be computed by taking derivatives of the sufficient statistic and the log-partition function . Example: the gamma distribution The gamma distribution is an exponential family with two parameters, \alpha and \beta. The likelihood function is \mathcal{L} (\alpha, \beta \mid x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}. Finding the maximum likelihood estimate of \beta for a single observed value x looks rather daunting. Its logarithm is much simpler to work with: \log \mathcal{L}(\alpha,\beta \mid x) = \alpha \log \beta - \log \Gamma(\alpha) + (\alpha-1) \log x - \beta x. \, To maximize the log-likelihood, we first take the partial derivative with respect to \beta: \frac{\partial \log \mathcal{L}(\alpha,\beta \mid x)}{\partial \beta} = \frac{\alpha}{\beta} - x. If there are a number of independent observations x_1, \ldots, x_n, then the joint log-likelihood will be the sum of individual log-likelihoods, and the derivative of this sum will be a sum of derivatives of each individual log-likelihood: \begin{align} & \frac{\partial \log \mathcal{L}(\alpha,\beta \mid x_1, \ldots, x_n)}{\partial \beta} \\ &= \frac{\partial \log \mathcal{L}(\alpha,\beta \mid x_1)}{\partial \beta} + \cdots + \frac{\partial \log \mathcal{L}(\alpha,\beta \mid x_n)}{\partial \beta} \\ &= \frac{n \alpha} \beta - \sum_{i=1}^n x_i. \end{align} To complete the maximization procedure for the joint log-likelihood, the equation is set to zero and solved for \beta: \widehat\beta = \frac{\alpha}{\bar{x}}. Here \widehat\beta denotes the maximum-likelihood estimate, and \textstyle \bar{x} = \frac{1}{n} \sum_{i=1}^n x_i is the sample mean of the observations. ==Background and interpretation==

Historical remarks The term "likelihood" has been in use in English since at least late Middle English. Its formal use to refer to a specific function in mathematical statistics was proposed by Ronald Fisher, in two research papers published in 1921 and 1922. His use of the term "likelihood" fixed the meaning of the term within mathematical statistics. A. W. F. Edwards (1972) established the axiomatic basis for use of the log-likelihood ratio as a measure of relative support for one hypothesis against another. The support function is then the natural logarithm of the likelihood function. Both terms are used in phylogenetics, but were not adopted in a general treatment of the topic of statistical evidence. Interpretations under different foundations Among statisticians, there is no consensus about what the foundation of statistics should be. There are four main paradigms that have been proposed for the foundation: frequentism, Bayesianism, likelihoodism, and AIC-based. For each of the proposed foundations, the interpretation of likelihood is different. The four interpretations are described in the subsections below. Frequentist interpretation Bayesian interpretation In Bayesian inference, although one can speak about the likelihood of any proposition or random variable given another random variable: for example the likelihood of a parameter value or of a statistical model (see marginal likelihood), given specified data or other evidence, the likelihood function remains the same entity, with the additional interpretations of (i) a conditional density of the data given the parameter (since the parameter is then a random variable) and (ii) a measure or amount of information brought by the data about the parameter value or even the model. Due to the introduction of a probability structure on the parameter space or on the collection of models, it is possible that a parameter value or a statistical model have a large likelihood value for given data, and yet have a low probability, or vice versa. Following Bayes' rule, the likelihood when seen as a conditional density can be multiplied by the prior probability density of the parameter and then normalized, to give a posterior probability density. == See also ==