Halley's method

Halley's method is a numerical algorithm for solving the nonlinear equation In this case, the function has to be a function of one real variable. The method consists of a sequence of iterations: : x_{n+1} = x_n - \frac{\ f(x_n)\ f'(x_n)\ }{\ \left[\ f'(x_n)\ \right]^2 - \tfrac{1}{2}\ f(x_n)\ f''(x_n)\ } beginning with an initial guess . If is a three times continuously differentiable function and is a zero of but not of its derivative, then, in a neighborhood of , the iterates satisfy: :| x_{n+1} - a | \le K \cdot ^3, \quad \text{ for some }\quad K > 0 ~. This means that the iterates converge to the zero if the initial guess is sufficiently close, and that the convergence is cubic. The following alternative formulation shows the similarity between Halley's method and Newton's method. The ratio \ f(x_n)/f'(x_n)\ only needs to be computed once, and this form is particularly useful when the other ratio, \ f''(x_n)/f'(x_n)\ , can be reduced to a simpler form: :x_{n+1}\ =\ x_n -\frac{ f(x_n) }{\ f'(x_n) - \frac{ f(x_n) }{\ f'(x_n)\ }\frac{\ f''(x_n)\ }{2}\ }\ =\ x_n - \frac{ f(x_n) }{\ f'(x_n)\ } \left[\ 1\ -\ \frac{1}{2} \cdot \frac{f(x_n)}{\ f'(x_n)\ } \cdot \frac{\ f''(x_n)\ }{ f'(x_n) }\ \right]^{-1} ~. When the second derivative, \ f''(x_n)\ , is very close to zero, the Halley's method iteration is almost the same as the Newton's method iteration. ==Motivation==

When deriving Newton's method, a proof starts with the approximation 0 = f(x_{n+1}) \approx f(x_n) + f'(x_n) (x_{n+1} - x_n) to compute x_{n+1}-x_n = -\frac{f(x_n)}{f'(x_n)}\,. Similarly for Halley's method, a proof starts with 0 = f(x_{n+1}) \approx f(x_n) + f'(x_n) (x_{n+1} - x_n) + \frac{f''(x_n)}2 (x_{n+1} - x_n)^2\,. For Halley's rational method, this is rearranged to give x_{n+1}-x_n = -\frac{f(x_n)}{f'(x_n) + \frac{f''(x_n)}2 (x_{n+1}-x_n)}\, where appears on both sides of the equation. Substituting in the Newton's method value for into the right-hand side of this last formula gives the formula for Halley's method, x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n) - \frac{f''(x_n)f(x_n)}{2f'(x_n)}}\,. Also see the motivation and proofs for the more general class of Householder's methods. ==Cubic convergence==

Suppose a is a root of f but not of its derivative. And suppose that the third derivative of f exists and is continuous in a neighborhood of a and is in that neighborhood. Then Taylor's theorem implies: :0 = f(a) = f(x_n) + f'(x_n) (a - x_n) + \frac{f(x_n)}{2} (a - x_n)^2 + \frac{f'(\xi)}{6} (a - x_n)^3 and also :0 = f(a) = f(x_n) + f'(x_n) (a - x_n) + \frac{f''(\eta)}{2} (a - x_n)^2, where ξ and η are numbers lying between a and . Multiply the first equation by 2f'(x_n) and subtract from it the second equation times f''(x_n)(a - x_n) to give: :\begin{align} 0 &= 2 f(x_n) f'(x_n) + 2 [f'(x_n)]^2 (a - x_n) + f'(x_n) f''(x_n) (a - x_n)^2 + \frac{f'(x_n) f'''(\xi)}{3} (a - x_n)^3 \\ &\qquad- f(x_n) f''(x_n) (a - x_n) - f'(x_n) f(x_n) (a - x_n)^2 - \frac{f(x_n) f''(\eta)}{2} (a - x_n)^3. \end{align} Canceling f'(x_n) f''(x_n) (a - x_n)^2 and re-organizing terms yields: :0 = 2 f(x_n) f'(x_n) + \left(2 [f'(x_n)]^2 - f(x_n) f''(x_n) \right) (a - x_n) + \left(\frac{f'(x_n) f'(\xi)}{3} - \frac{f(x_n) f''(\eta)}{2} \right) (a - x_n)^3. Put the second term on the left side and divide through by : 2 [f'(x_n)]^2 - f(x_n) f''(x_n) to get: :a - x_n = \frac{-2f(x_n) f'(x_n)}{2[f'(x_n)]^2 - f(x_n) f''(x_n)} - \frac{2f'(x_n) f'(\xi) - 3 f(x_n) f''(\eta)}{6(2 [f'(x_n)]^2 - f(x_n) f''(x_n))} (a - x_n)^3. Thus: :a - x_{n+1} = - \frac{2 f'(x_n) f'(\xi) - 3 f(x_n) f''(\eta)}{12[f'(x_n)]^2 - 6 f(x_n) f''(x_n)} (a - x_n)^3. The limit of the coefficient on the right side as is: :-\frac{2 f'(a) f'(a) - 3 f(a) f''(a)}{12 [f'(a)]^2 - 6 f(a) f''(a)}. If we take K to be a little larger than the absolute value of this, we can take absolute values of both sides of the formula and replace the absolute value of coefficient by its upper bound near a to get: :|a - x_{n+1}| \leq K |a - x_n|^3 which is what was to be proved. To summarize, :\Delta x_{i+1} =\frac{3(f'')^2 - 2f' f'''}{12(f')^2} (\Delta x_i)^3 + O[\Delta x_i]^4, \qquad \Delta x_i \triangleq x_i - a. ==Relation to Newton's method==

Halley's rational method applied to the real-valued function is the same as applying Newton's method x_{n+1} = x_n - \frac{g(x_n)}{g'(x_n)} to find the zeros of the function g(x) = \frac{f(x)}{\sqrt}\,, where is any non-zero constant. Applying Newton's method to will have cubic convergence (or better), whereas Newton's method applied directly to will usually have quadratic convergence. For example, using Halley's method to find a zero of , which is useful for efficiently and precisely estimating , is the same as using Newton's method to find a zero of . ==Example==

Use of Halley's method to compute square roots Just as Newton's method can be used to compute square roots, Halley's method can be specialized for the same purpose with cubic convergence. To find the positive square root of a given number , consider the function :f(x) = x^2 - S, \quad f'(x) = 2x, \quad f''(x) = 2. Substituting these into the general form of Halley's iteration, : x_{n+1} = x_n - \frac{2 f(x_n) f'(x_n)}{2 [f'(x_n)]^2 - f(x_n) f''(x_n)}, gives : x_{n+1} = x_n \cdot \frac{x_n^2 + 3S}{3x_n^2 + S}. This is known as the rational form of Halley’s method for square roots. It requires only arithmetic operations—no square roots—and converges cubically to \sqrt{S} for any positive initial guess x_0 > 0. ==Halley's irrational method==

Halley actually developed two third-order root-finding methods. The above, using only a division, is referred to as ''Halley's rational method''. A second, "irrational" method uses a square root as well. It starts with :f(x_{n+1}) \approx f(x_n) + f'(x_n)(x_{n+1} - x_{n}) + \frac{f''(x_n)}{2}(x_{n+1} - x_{n})^2 and solves f(x_{n+1}) \approx 0 for the value (x_{n+1} - x_{n}) using the form of the quadratic formula for a(x_{n+1}-x_n)^2 + b(x_{n+1}-x_n) + c = 0 that has the radical in the denominator, :x_{n+1} = x_n - \frac{2c}{b\left(1 + \sqrt{1-4ac/b^2}\right)} = x_n - \frac{2f(x_n)}{f'(x_n) \left( 1 + \sqrt{1 - \frac{2f(x_n)f''(x_n)}{f'(x_n)^2}}\right)}. The sign for the radical is chosen so that x_{n+1} is the root nearer to x_{n}. This iteration was "deservedly preferred" to the rational method by Halley on the grounds that it tends to have about half of the error of the rational method, a benefit which multiplies as it is iterated. This formulation reduces to Halley's rational method under the approximation that {{tmath|\sqrt{1-z} \approx 1 - z/2}}. Muller's method could be considered as modification of this method. So, this method can be used to find the complex roots. ==Multiple variables==

Halley's method has been adapted to multiple variables, where the goal is to find a root \vec{x} of \vec{f}(\vec{x}) = \vec{0} for some function f \colon \R^n \rightarrow \R^n with positive integer . In this case, Newton's method uses the constant and linear terms of the multivariate Taylor series, and solves the equation \vec{0} = \vec{f}(\vec{x}_{n+1}) \approx \vec{f}(\vec{x}_n) + M (\vec{x}_{n+1}-\vec{x}_n) with matrix M_{ij} = \left.\frac{\partial f_i}{\partial x_j}\right|_{\vec{x}=\vec{x}_n} to give \vec{x}_{n+1} = \vec{x}_n - M^{-1} \vec{f}(\vec{x}_n) using matrix inversion. Following the motivation for the one-variable case, Halley's method gives \vec{x}_{n+1} = \vec{x}_n - \left(M-\frac{1}{2} \sum_k \frac{\partial M}{\partial x_k} \left(M^{-1}\vec{f}(\vec{x}_n)\right)_k\right)^{-1} \vec{f}(\vec{x}_n) where the subscript indicates the -th coordinate of the associated vector. ==References==