Add an option to provide the sparsity pattern of jacobians and hessians

[amontoison]

@jbcaillau

[jbcaillau]

@jbcaillau

Thanks @amontoison So, provide:

• sparse AD Jacobian (forward)
• sparse AD Hessian (forward and reverse) ?

[amontoison]

@jbcaillau

Thanks @amontoison So, provide:

* sparse AD Jacobian (forward)

* sparse AD Hessian (forward and reverse) ?

@jbcaillau
The sparsity pattern of Jacobians and Hessians is not related to AD but we must store it for the decompression step.
The AD is just used to recover the value of the nnz coefficients and we can use what we want for the backend.
The supported sparse backend are:

• SparseADJacobian -- Forward (Jacobian);
• SparseADHessian -- Forward over forward (Hessian);
• SparseReverseADHessian -- Forward over Reverse (Hessian).

In the future I would like to only support two backends (SparseADJacobian and SparseADHessian) with options for the "mode" (forward, reverse, both, forward over reverse, ...) and the "AD package" (Enzyme.jl, ForwardDiff.jl, ReverseDiff.jl, ...).

[github-actions]

Package name	latest	stable
CaNNOLeS.jl
DCISolver.jl
DerivativeFreeSolvers.jl
JSOSolvers.jl
NLPModelsIpopt.jl
OptimalControl.jl
OptimizationProblems.jl
Percival.jl
QuadraticModels.jl
SolverBenchmark.jl
SolverTools.jl

[jbcaillau]

@jbcaillau The sparsity pattern of Jacobians and Hessians is not related to AD but we must store it for the decompression step. The AD is just used to recover the value of the nnz coefficients and we can use what we want for the backend [...]

@amontoison OK. Thought it was more directly related to this request (example / tuto on how to build an NLPModel)
control-toolbox/CTDirect.jl#183

[tmigot]

Thanks @amontoison ! Just a small improvement, I don't think we need the parametric type anymore.

[tmigot]

After, this PR I can rehabilitate this PR #209 to have such tutorials

[amontoison]

@tmigot Do you think that we can add an option when we create an ADNLPModel / ADNLSModel to provide a sparsity pattern?
An option like pattern_hessian / sp_hessian.

[tmigot]

kwargs given to the ADNLPModels constructor are passed to the ADModelBackend which are passed to each backend.
Would there be a way to modify this PR so that it works?
Maybe add the sparsity pattern as kwargs with default of size (0,0) and then avoid calling the function only if size is different something like this?

I am not against adding kwargs, but ADNLPModels has so many constructors, I would love not adding more code if possible.

[amontoison]

Ok, let's not add new keywords and just add an example / tutorial in the documentation to provide the sparsity patterns.