Add methods that require derivatives

[jacobwilliams]

e.g, Newton-Raphson, Halley.

[jacobwilliams]

See also #19. Maybe we keep this library focused as is?

[ivan-pi]

What are some of the design points for derivative-based root-finding methods?

• Methods
  • Newton-Raphson
  • Halley
  • Schröder
  • Secant method and Steffensen's method (using the derivatives of the initial guess to start the iteration sequence)
• Callback function design (see also Callback function design grimme-lab/nlopt-f#16)
• Bracketing and fallback bisection
• Termination conditions
• Automatic differentiation for derivative evaluation (complex- or dual-number based approaches)
• Routines for checking consistency of first and second derivatives

I think two or three if not more items in the list have overlap with this library, and could benefit from reuse.

The first question to answer is perhaps if the derivative-based methods would go to a separate module or directly into root_module? (They can be implemented in a submodule of course.)

Personally I think it would be useful to have some flexibility:

• root_module (top-level model exposing everything)
  • root_derivative_module (derivative-based methods, consistency checking)
  • root_bracket_module (function-only methods present currently)

Using the fpm`s tree-shaking mechanism, a user wanting to keep his compile time as low as possible, could import only the specific module instead of the top-level one.

GSL puts the function-only, and derivative based methods under the same <gsl/gsl_roots.h> header; also the Boost Math Toolkit includes derivative-based solvers in the same header <boost/math/tools/roots.hpp> as the function-only methods. SciPy includes them under the same interface scipy.optimize.root_scalar using the fprime and fprimer2 arguments:

def scipy.optimize.root_scalar(f, args=(), method=None, bracket=None, 
    fprime=None, fprime2=None, x0=None, x1=None, 
    xtol=None, rtol=None, maxiter=None, options=None):
    ...

Based on this I'd argue that users will find it easier to have these in the same package. Also aspects such as the documentation, tutorials, and distribution mechanism can be unified.