Pontryagin's maximum principle

For set \mathcal{U} and functions :\Psi : \reals^n \to \reals, :H : \reals^n \times \mathcal{U} \times \reals^n \times \reals \to \reals, :L : \reals^n \times \mathcal{U} \to \reals, :f : \reals^n \times \mathcal{U} \to \reals^n, we use the following notation: : \Psi_T(x(T))= \left.\frac{\partial \Psi(x)}{\partial T}\right|_{x=x(T)} \, , : \Psi_x(x(T))=\begin{bmatrix} \left.\frac{\partial \Psi(x)}{\partial x_1}\right|_{x=x(T)} & \cdots & \left.\frac{\partial \Psi(x)}{\partial x_n} \right|_{x=x(T)} \end{bmatrix} , : H_x(x^*,u^*,\lambda^*,t)=\begin{bmatrix} \left.\frac{\partial H}{\partial x_1}\right|_{x=x^*,u=u^*,\lambda=\lambda^*} & \cdots & \left.\frac{\partial H}{\partial x_n}\right|_{x=x^*,u=u^*,\lambda=\lambda^*} \end{bmatrix} , : L_x(x^*,u^*)=\begin{bmatrix} \left.\frac{\partial L}{\partial x_1}\right|_{x=x^*,u=u^*} & \cdots & \left.\frac{\partial L}{\partial x_n}\right|_{x=x^*,u=u^*} \end{bmatrix} , : f_x(x^*,u^*)=\begin{bmatrix} \left.\frac{\partial f_1}{\partial x_1}\right|_{x=x^*,u=u^*} & \cdots & \left.\frac{\partial f_1}{\partial x_n}\right|_{x=x^*,u=u^*} \\ \vdots & \ddots & \vdots \\ \left.\frac{\partial f_n}{\partial x_1}\right|_{x=x^*,u=u^*} & \ldots & \left.\frac{\partial f_n}{\partial x_n}\right|_{x=x^*,u=u^*} \end{bmatrix} . ==Formal statement of necessary conditions for minimization problems==

Here the necessary conditions are shown for minimization of a functional. Consider an n-dimensional dynamical system, with state variable x \in \R^n, and control variable u \in \mathcal{U}, where \mathcal{U} is the set of admissible controls. The evolution of the system is determined by the state and the control, according to the differential equation \dot{x}=f(x,u). Let the system's initial state be x_0 and let the system's evolution be controlled over the time-period with values t \in [0, T]. The latter is determined by the following differential equation: : \dot{x}=f(x,u), \quad x(0)=x_0, \quad u(t) \in \mathcal{U}, \quad t \in [0,T] The control trajectory u: [0, T] \to \mathcal{U} is to be chosen according to an objective. The objective is a functional J defined by : J=\Psi(x(T))+\int^T_0 L\big(x(t),u(t)\big) \,dt , where L(x, u) can be interpreted as the rate of cost for exerting control u in state x, and \Psi(x) can be interpreted as the cost for ending up at state x. The specific choice of L, \Psi depends on the application. The constraints on the system dynamics can be adjoined to the Lagrangian L by introducing time-varying Lagrange multiplier vector \lambda, whose elements are called the costates of the system. This motivates the construction of the Hamiltonian H defined for all t \in [0,T] by: : H\big(x(t),u(t),\lambda(t),t\big)=\lambda^{\rm T}(t)\cdot f\big(x(t),u(t)\big) + L\big(x(t),u(t)\big) where \lambda^{\rm T} is the transpose of \lambda. Pontryagin's minimum principle states that the optimal state trajectory x^*, optimal control u^*, and corresponding Lagrange multiplier vector \lambda^* must minimize the Hamiltonian H so that for all time t \in [0,T] and for all permissible control inputs u \in \mathcal{U}. Here, the trajectory of the Lagrangian multiplier vector \lambda is the solution to the costate equation: {{NumBlk|:| -\dot{\lambda}^{\rm T}(t)=H_x\big(x^*(t),u^*(t),\lambda(t),t\big)=\lambda^{\rm T}(t)\cdot f_x\big(x^*(t),u^*(t)\big) + L_x\big(x^*(t),u^*(t)\big)|}} The boundary conditions depend on if final state x(T) and final time T are fixed. These conditions are found by looking at how the functional J (differentially) varies when x(T) and T are (differentially) varied. Assuming the final state and time are independent of each other, x(T) not being fixed results in the condition {{NumBlk|:| \lambda^{\rm T}(T)=\Psi_x(x(T)), |}} and T not being fixed results in the condition These four conditions in (1)-(4) are the necessary conditions for an optimal control. ==See also==