Re: Algorithmic Differentiation (Using a C++ library)

[Oliver Heimlich]

(I hope you don't mind that I cc to the mailing list)

On 01.03.2017 21:07, Brad Bell wrote:
> On 03/01/2017 12:12 PM, Oliver Heimlich wrote:
>> On 01.03.2017 15:38, Brad Bell wrote:
>>> Another thing to consider, instead of the library, is using CppAD +
>>> Eigen. That would make it easy to implement matrix operations. For
>>> example, see
>>>      https://www.coin-or.org/CppAD/Doc/eigen_array.cpp.htm
>> Slightly off topic: There once was an Octave package for AD [1], which
>> is now unmaintained. I believe the reason is that it was mainly build in
>> CPP code, which is no longer compatible with the current Octave API.
>> Most of the code is used to introduce a new data type into Octave, with
>> overloaded operations.
>>
>> Now that we have classes in Octave, it would be possible to revive this
>> package using a classdef data type, overloaded operators and vectorized
>> m-file code. (Possible GSoC project?!?
> I have taken a brief look at the package source code. It appears to just
> implement first order foward mode. Is this correct ?

Yes, this is also my impression.

> There is a lot to and AD package (e.g, Cppad and Adolc) that can compute
> arbitrary order derivatives using forward or reverse mode and  combine
> multiple rows or columns when computing sparse Jacobians and Hessians .
> While this, and other features of a complete AD package can be
> reproduced, I think it would be simpler to take advantage of an existing
> package.

First order forward mode is quite simple to implement. I am not an
expert in AD and wasn't aware of the advanced functionality that you
want to use.

> Yes I agree that, hooking up and AD package to Octave would make a good
> GSoC project.


>> The unmaintained package contains documentation and many test cases,
>> which could easily be taken over.
> As for a source of AD test cases, one could look here
>     https://www.coin-or.org/CppAD/Doc/listallexamples.htm
> For more tests, that are not also examples, see
>     https://github.com/coin-or/CppAD/tree/master/test_more

Test cases of the Octave package can be found in [2]

>> If everything goes well, we would have:
>> - fast code that uses Octave's matrix operations
>> - easier to maintain m-file code
>> - the possibility to use other data types in AD

> It might also help to look at the MATLAB packages that do AD; see
>     http://www.autodiff.org/?module=Tools&language=MATLAB

As far as I see, these packages are either proprietary software or use
the source transformation method (instead of operator overloading).

>> Regarding the last point: I think of combining AD with the interval data
>> type. This way it would become possible to compute guaranteed enclosures
>> for the Jacobian, and you don't have to rely on “NearEqual” checks with
>> approximate values in your script.
> Cppad allows for an arbitrary Base type that the derivatives a
> calculated with. It can be an interval type.

I am not sure, if (and how easy) it is possibly to combine a
templated(?) C++ library with types that are implemented in Octave as
classes.

Oliver

>> [1] https://octave.sourceforge.io/ad/
[2]
https://sourceforge.net/p/octave/automatic_differentiation/ci/default/tree/inst/__ga__.m