Functional derivative

In this section, the functional differential (or variation or first variation) is defined. Then the functional derivative is defined in terms of the functional differential. Functional differential Suppose B is a Banach space and F is a functional defined on B. The differential of F at a point \rho\in B is the linear functional \delta F[\rho,\cdot] on B defined by the condition that, for all \phi\in B, F[\rho+\phi] - F[\rho] = \delta F [\rho; \phi] + \varepsilon \left\|\phi\right\| where \varepsilon is a real number that depends on \|\phi\| in such a way that \varepsilon\to 0 as \|\phi\|\to 0. This means that \delta F[\rho,\cdot] is the Fréchet derivative of F at \rho. However, this notion of functional differential is so strong it may not exist, and in those cases a weaker notion, like the Gateaux derivative is preferred. In many practical cases, the functional differential is defined as the directional derivative \begin{align} \delta F[\rho,\phi] &= \lim_{\varepsilon\to 0}\frac{F[\rho+\varepsilon \phi]-F[\rho]}{\varepsilon} \\[1ex] &= \left [ \frac{d}{d\varepsilon}F[\rho+\varepsilon \phi]\right ]_{\varepsilon=0}. \end{align} Note that this notion of the functional differential can even be defined without a norm. In a more general case the function space B appearing as the domain of F is not a vector space, and therefore variations of the form \rho + \varepsilon \phi do not make sense. In this case we consider a variation \alpha_{?} : (-\varepsilon_0, \varepsilon_0) \to B of \rho to be a C^1-family of functions such that \alpha_0 = \rho. Denoting the space of all such variations as \mathcal V_\rho, the functional differential \delta F[\rho] : \mathcal V_\rho \to \mathbb R is the functional \begin{align} \delta F[\rho;\alpha] = \delta F[\rho][\alpha] = \lim_{ \epsilon \to 0 } \frac{F[\alpha_{\epsilon}] - F[\rho]}{\epsilon} = F[\alpha_{?}]'(0) \end{align} where F[\alpha_?](\epsilon) = F[\alpha_\epsilon] . The above then becomes the special case \alpha_\epsilon = \rho + \epsilon \eta . Functional derivative In many applications, the domain of the functional F is a space of differentiable functions \rho defined on some space \Omega and F is of the form F[\rho] = \int_\Omega L(x,\rho(x),D\rho(x))\,dx for some function L(x,\rho(x),D\rho(x)) that may depend on x, the value \rho(x) and the derivative D\rho(x). If this is the case and, moreover, \delta F[\rho,\phi] can be written as the integral of \phi times another function (denoted ) \delta F [\rho, \phi] = \int_\Omega \frac {\delta F} {\delta \rho}(x) \ \phi(x) \ dx then this function is called the functional derivative of at . If F is restricted to only certain functions \rho (for example, if there are some boundary conditions imposed) then \phi is restricted to functions such that \rho+\varepsilon\phi continues to satisfy these conditions. Heuristically, \phi is the change in \rho, so we 'formally' have \phi = \delta\rho, and then this is similar in form to the total differential of a function F(\rho_1,\rho_2,\dots,\rho_n), dF = \sum_{i=1} ^n \frac {\partial F} {\partial \rho_i} \ d\rho_i , where \rho_1,\rho_2,\dots,\rho_n are independent variables. Comparing the last two equations, the functional derivative \frac{\delta F}{\delta\rho(x)} has a role similar to that of the partial derivative \frac{\partial F}{\partial\rho_i}, where the variable of integration x is like a continuous version of the summation index i. One thinks of as the gradient of at the point , so the value measures how much the functional will change if the function is changed at the point . Hence the formula \int \frac{\delta F}{\delta\rho}(x) \phi(x) \; dx is regarded as the directional derivative at point \rho in the direction of \phi. This is analogous to vector calculus, where the inner product of a vector v with the gradient gives the directional derivative in the direction of v. ==Properties==

Like the derivative of a function, the functional derivative satisfies the following properties, where and are functionals: • Linearity: \frac{\delta(\lambda F + \mu G)[\rho ]}{\delta \rho(x)} = \lambda \frac{\delta F[\rho]}{\delta \rho(x)} + \mu \frac{\delta G[\rho]}{\delta \rho(x)}, where are constants. • Product rule: \frac{\delta(FG)[\rho]}{\delta \rho(x)} = \frac{\delta F[\rho]}{\delta \rho(x)} G[\rho] + F[\rho] \frac{\delta G[\rho]}{\delta \rho(x)} \, , • Chain rules: • If is a functional and another functional, then \frac{\delta F[G[\rho }{\delta\rho(y)} = \int dx \frac{\delta F[G]}{\delta G(x)}_{G = G[\rho]}\cdot\frac {\delta G[\rho](x)} {\delta\rho(y)} \ . • If is an ordinary differentiable function (local functional) , then this reduces to \frac{\delta F[g(\rho)] }{\delta\rho(y)} = \frac{\delta F[g(\rho)]}{\delta g[\rho(y) ]} \ \frac {dg(\rho)} {d\rho(y)} \ . ==Determining functional derivatives==

A formula to determine functional derivatives for a common class of functionals can be written as the integral of a function and its derivatives. This is a generalization of the Euler–Lagrange equation: indeed, the functional derivative was introduced in physics within the derivation of the Lagrange equation of the second kind from the principle of least action in Lagrangian mechanics (18th century). The first three examples below are taken from density functional theory (20th century), the fourth from statistical mechanics (19th century). Formula Given a functional F[\rho] = \int f( \boldsymbol{r}, \rho(\boldsymbol{r}), \nabla\rho(\boldsymbol{r}) )\, d\boldsymbol{r}, and a function \phi(\boldsymbol{r}) that vanishes on the boundary of the region of integration, from a previous section Definition, \begin{align} \int \frac{\delta F}{\delta\rho(\boldsymbol{r})} \, \phi(\boldsymbol{r}) \, d\boldsymbol{r} & = \left [ \frac{d}{d\varepsilon} \int f( \boldsymbol{r}, \rho + \varepsilon \phi, \nabla\rho+\varepsilon\nabla\phi )\, d\boldsymbol{r} \right ]_{\varepsilon=0} \\ & = \int \left( \frac{\partial f}{\partial\rho} \, \phi + \frac{\partial f}{\partial\nabla\rho} \cdot \nabla\phi \right) d\boldsymbol{r} \\ & = \int \left[ \frac{\partial f}{\partial\rho} \, \phi + \nabla \cdot \left( \frac{\partial f}{\partial\nabla\rho} \, \phi \right) - \left( \nabla \cdot \frac{\partial f}{\partial\nabla\rho} \right) \phi \right] d\boldsymbol{r} \\ & = \int \left[ \frac{\partial f}{\partial\rho} \, \phi - \left( \nabla \cdot \frac{\partial f}{\partial\nabla\rho} \right) \phi \right] d\boldsymbol{r} \\ & = \int \left( \frac{\partial f}{\partial\rho} - \nabla \cdot \frac{\partial f}{\partial\nabla\rho} \right) \phi(\boldsymbol{r}) \ d\boldsymbol{r} \, . \end{align} The second line is obtained using the total derivative, where is a derivative of a scalar with respect to a vector. The third line was obtained by use of a product rule for divergence. The fourth line was obtained using the divergence theorem and the condition that \phi=0 on the boundary of the region of integration. Since \phi is also an arbitrary function, applying the fundamental lemma of calculus of variations to the last line, the functional derivative is \frac{\delta F}{\delta\rho(\boldsymbol{r})} = \frac{\partial f}{\partial\rho} - \nabla \cdot \frac{\partial f}{\partial\nabla\rho} where and . This formula is for the case of the functional form given by at the beginning of this section. For other functional forms, the definition of the functional derivative can be used as the starting point for its determination. (See the example Coulomb potential energy functional.) The above equation for the functional derivative can be generalized to the case that includes higher dimensions and higher order derivatives. The functional would be, F[\rho(\boldsymbol{r})] = \int f\left( \boldsymbol{r}, \rho(\boldsymbol{r}), \nabla\rho(\boldsymbol{r}), \nabla^{(2)}\rho(\boldsymbol{r}), \dots, \nabla^{(N)}\rho(\boldsymbol{r})\right)\, d\boldsymbol{r}, where the vector , and is a tensor whose components are partial derivative operators of order , \left [ \nabla^{(i)} \right ]_{\alpha_1 \alpha_2 \cdots \alpha_i} = \frac {\partial^{\, i}} {\partial r_{\alpha_1} \partial r_{\alpha_2} \cdots \partial r_{\alpha_i} } \qquad \qquad \text{where} \quad \alpha_1, \alpha_2, \dots, \alpha_i = 1, 2, \dots , n \ . An analogous application of the definition of the functional derivative yields \begin{align} \frac{\delta F[\rho]}{\delta \rho} &{} = \frac{\partial f}{\partial\rho} - \nabla \cdot \frac{\partial f}{\partial(\nabla\rho)} + \nabla^{(2)} \cdot \frac{\partial f}{\partial\left(\nabla^{(2)}\rho\right)} + \dots + (-1)^N \nabla^{(N)} \cdot \frac{\partial f}{\partial\left(\nabla^{(N)}\rho\right)} \\ &{} = \frac{\partial f}{\partial\rho} + \sum_{i=1}^N (-1)^{i}\nabla^{(i)} \cdot \frac{\partial f}{\partial\left(\nabla^{(i)}\rho\right)} \ . \end{align} In the last two equations, the components of the tensor \frac{\partial f}{\partial\left(\nabla^{(i)}\rho\right)} are partial derivatives of with respect to partial derivatives of ρ, \left [ \frac {\partial f} {\partial \left (\nabla^{(i)}\rho \right ) } \right ]_{\alpha_1 \alpha_2 \cdots \alpha_i} = \frac {\partial f} {\partial \rho_{\alpha_1 \alpha_2 \cdots \alpha_i} } where \rho_{\alpha_1 \alpha_2 \cdots \alpha_i} \equiv \frac {\partial^{\,i}\rho} {\partial r_{\alpha_1} \, \partial r_{\alpha_2} \cdots \partial r_{\alpha_i} } , and the tensor scalar product is, \nabla^{(i)} \cdot \frac{\partial f}{\partial\left(\nabla^{(i)}\rho\right)} = \sum_{\alpha_1, \alpha_2, \cdots, \alpha_i = 1}^n \ \frac {\partial^{\, i} } {\partial r_{\alpha_1} \, \partial r_{\alpha_2} \cdots \partial r_{\alpha_i} } \ \frac {\partial f} {\partial \rho_{\alpha_1 \alpha_2 \cdots \alpha_i} } \ . Examples Thomas–Fermi kinetic energy functional The Thomas–Fermi model of 1927 used a kinetic energy functional for a noninteracting uniform electron gas in a first attempt of density-functional theory of electronic structure: T_\mathrm{TF}[\rho] = C_\mathrm{F} \int \rho^\frac53(\mathbf{r}) \, d\mathbf{r} \, . Since the integrand of does not involve derivatives of , the functional derivative of is, \frac{\delta T_{\mathrm{TF}}}{\delta \rho (\boldsymbol{r}) } = C_\mathrm{F} \frac{\partial \rho^\frac53(\mathbf{r})}{\partial \rho(\mathbf{r})} = \frac{5}{3} C_\mathrm{F} \rho^\frac23(\mathbf{r}) \, . Coulomb potential energy functional The electron–nucleus potential energy is V[\rho] = \int \frac{\rho(\boldsymbol{r})} \ d\boldsymbol{r}. Applying the definition of functional derivative, \begin{align} \int \frac{\delta V}{\delta \rho(\boldsymbol{r})} \ \phi(\boldsymbol{r}) \ d\boldsymbol{r} & {} = \left [ \frac{d}{d\varepsilon} \int \frac{\rho(\boldsymbol{r}) + \varepsilon \phi(\boldsymbol{r})} \ d\boldsymbol{r} \right ]_{\varepsilon=0} \\[1ex] & {} = \int \frac {\phi(\boldsymbol{r})} \ d\boldsymbol{r} \, . \end{align} So, \frac{\delta V}{\delta \rho(\boldsymbol{r})} = \frac{1} \ . The functional derivative of the classical part of the electron–electron interaction (often called Hartree energy) is J[\rho] = \frac{1}{2}\iint \frac{\rho(\mathbf{r}) \rho(\mathbf{r}')}\, d\mathbf{r} d\mathbf{r}' \, . From the definition of the functional derivative, \begin{align} \int \frac{\delta J}{\delta\rho(\boldsymbol{r})} \phi(\boldsymbol{r})d\boldsymbol{r} & {} = \left [ \frac {d \ }{d\varepsilon} \, J[\rho + \varepsilon\phi] \right ]_{\varepsilon = 0} \\ & {} = \left [ \frac {d \ }{d\varepsilon} \, \left ( \frac{1}{2}\iint \frac {[\rho(\boldsymbol{r}) + \varepsilon \phi(\boldsymbol{r})] \, [\rho(\boldsymbol{r}') + \varepsilon \phi(\boldsymbol{r}')] }\, d\boldsymbol{r} d\boldsymbol{r}' \right ) \right ]_{\varepsilon = 0} \\ & {} = \frac{1}{2}\iint \frac {\rho(\boldsymbol{r}') \phi(\boldsymbol{r}) }\, d\boldsymbol{r} d\boldsymbol{r}' + \frac{1}{2}\iint \frac {\rho(\boldsymbol{r}) \phi(\boldsymbol{r}') }\, d\boldsymbol{r} d\boldsymbol{r}' \\ \end{align} The first and second terms on the right hand side of the last equation are equal, since and in the second term can be interchanged without changing the value of the integral. Therefore, \int \frac{\delta J}{\delta\rho(\boldsymbol{r})} \phi(\boldsymbol{r})d\boldsymbol{r} = \int \left ( \int \frac {\rho(\boldsymbol{r}') } d\boldsymbol{r}' \right ) \phi(\boldsymbol{r}) d\boldsymbol{r} and the functional derivative of the electron-electron Coulomb potential energy functional [ρ] is, \frac{\delta J}{\delta\rho(\boldsymbol{r})} = \int \frac {\rho(\boldsymbol{r}') } d\boldsymbol{r}' \, . The second functional derivative is \frac{\delta^2 J[\rho]}{\delta \rho(\mathbf{r}')\delta\rho(\mathbf{r})} = \frac{\partial}{\partial \rho(\mathbf{r}')} \left ( \frac{\rho(\mathbf{r}')} \right ) = \frac{1}. von Weizsäcker kinetic energy functional In 1935 von Weizsäcker proposed to add a gradient correction to the Thomas-Fermi kinetic energy functional to make it better suit a molecular electron cloud: T_\mathrm{W}[\rho] = \frac{1}{8} \int \frac{\nabla\rho(\mathbf{r}) \cdot \nabla\rho(\mathbf{r})}{ \rho(\mathbf{r}) } d\mathbf{r} = \int t_\mathrm{W}(\mathbf{r}) \ d\mathbf{r} \, , where t_\mathrm{W} \equiv \frac{1}{8} \frac{\nabla\rho \cdot \nabla\rho}{ \rho } \qquad \text{and} \ \ \rho = \rho(\boldsymbol{r}) \ . Using a previously derived formula for the functional derivative, \begin{align} \frac{\delta T_\mathrm{W}}{\delta \rho} & = \frac{\partial t_\mathrm{W}}{\partial \rho} - \nabla\cdot\frac{\partial t_\mathrm{W}}{\partial \nabla \rho} \\ & = -\frac{1}{8}\frac{\nabla\rho \cdot \nabla\rho}{\rho^2} - \left ( \frac {1}{4} \frac {\nabla^2\rho} {\rho} - \frac {1}{4} \frac {\nabla\rho \cdot \nabla\rho} {\rho^2} \right ) \qquad \text{where} \ \ \nabla^2 = \nabla \cdot \nabla \ , \end{align} and the result is, \frac{\delta T_\mathrm{W}}{\delta \rho} = \ \ \, \frac{1}{8}\frac{\nabla\rho \cdot \nabla\rho}{\rho^2} - \frac{1}{4}\frac{\nabla^2\rho}{\rho} \ . Entropy The entropy of a discrete random variable is a functional of the probability mass function. H[p(x)] = -\sum_x p(x) \log p(x) Thus, \begin{align} \sum_x \frac{\delta H}{\delta p(x)} \, \phi(x) & {} = \left[ \frac{d}{d\varepsilon} H[p(x) + \varepsilon\phi(x)] \right]_{\varepsilon=0}\\ & {} = \left [- \, \frac{d}{d\varepsilon} \sum_x \, [p(x) + \varepsilon\phi(x)] \ \log [p(x) + \varepsilon\phi(x)] \right]_{\varepsilon=0} \\ & {} = -\sum_x \, [1+\log p(x)] \ \phi(x) \, . \end{align} Thus, \frac{\delta H}{\delta p(x)} = -1-\log p(x). Exponential Let F[\varphi(x)]= e^{\int \varphi(x) g(x)dx}. Using the delta function as a test function, \begin{align} \frac{\delta F[\varphi(x)]}{\delta \varphi(y)} & {} = \lim_{\varepsilon\to 0}\frac{F[\varphi(x)+\varepsilon\delta(x-y)]-F[\varphi(x)]}{\varepsilon}\\ & {} = \lim_{\varepsilon\to 0}\frac{e^{\int (\varphi(x)+\varepsilon\delta(x-y)) g(x)dx}-e^{\int \varphi(x) g(x)dx}}{\varepsilon}\\ & {} = e^{\int \varphi(x) g(x)dx}\lim_{\varepsilon\to 0}\frac{e^{\varepsilon \int \delta(x-y) g(x)dx}-1}{\varepsilon}\\ & {} = e^{\int \varphi(x) g(x)dx}\lim_{\varepsilon\to 0}\frac{e^{\varepsilon g(y)}-1}{\varepsilon}\\ & {} = e^{\int \varphi(x) g(x)dx}g(y). \end{align} Thus, \frac{\delta F[\varphi(x)]}{\delta \varphi(y)} = g(y) F[\varphi(x)]. This is particularly useful in calculating the correlation functions from the partition function in quantum field theory. Functional derivative of a function A function can be written in the form of an integral like a functional. For example, \rho(\boldsymbol{r}) = F[\rho] = \int \rho(\boldsymbol{r}') \delta(\boldsymbol{r}-\boldsymbol{r}')\, d\boldsymbol{r}'. Since the integrand does not depend on derivatives of ρ, the functional derivative of ρ is, \frac {\delta \rho(\boldsymbol{r})} {\delta\rho(\boldsymbol{r}')} \equiv \frac {\delta F} {\delta\rho(\boldsymbol{r}')} = \frac{\partial \ \ }{\partial \rho(\boldsymbol{r}')} \, [\rho(\boldsymbol{r}') \delta(\boldsymbol{r}-\boldsymbol{r}')] = \delta(\boldsymbol{r}-\boldsymbol{r}'). Functional derivative of iterated function The functional derivative of the iterated function f(f(x)) is given by: \frac{\delta f(f(x))}{\delta f(y) } = f'(f(x))\delta(x-y) + \delta(f(x)-y) and \frac{\delta f(f(f(x)))}{\delta f(y) } = f'(f(f(x))(f'(f(x))\delta(x-y) + \delta(f(x)-y)) + \delta(f(f(x))-y) In general: \frac{\delta f^N(x)}{\delta f(y)} = f'\left( f^{N-1}(x) \right) \frac{ \delta f^{N-1}(x)}{\delta f(y)} + \delta\left( f^{N-1}(x) - y \right) Putting in gives: \frac{\delta f^{-1}(x)}{\delta f(y) } = - \frac{ \delta\left(f^{-1}(x)-y \right) }{ f'\left(f^{-1}(x)\right) } ==Using the delta function as a test function==

In physics, it is common to use the Dirac delta function \delta(x-y) in place of a generic test function \phi(x), for yielding the functional derivative at the point y (this is a point of the whole functional derivative as a partial derivative is a component of the gradient): \frac{\delta F[\rho(x)]}{\delta \rho(y)}=\lim_{\varepsilon\to 0}\frac{F[\rho(x)+\varepsilon\delta(x-y)]-F[\rho(x)]}{\varepsilon}. This works in cases when F[\rho(x)+\varepsilon f(x)] formally can be expanded as a series (or at least up to first order) in \varepsilon. The formula is however not mathematically rigorous, since F[\rho(x)+\varepsilon\delta(x-y)] is usually not even defined. The definition given in a previous section is based on a relationship that holds for all test functions \phi(x), so one might think that it should hold also when \phi(x) is chosen to be a specific function such as the delta function. However, the latter is not a valid test function (it is not even a proper function). In the definition, the functional derivative describes how the functional F[\rho(x)] changes as a result of a small change in the entire function \rho(x). The particular form of the change in \rho(x) is not specified, but it should stretch over the whole interval on which x is defined. Employing the particular form of the perturbation given by the delta function has the meaning that \rho(x) is varied only in the point y. Except for this point, there is no variation in \rho(x). ==Notes==