Re: Algorithmic Differentiation (Using a C++ library)

[Joel Dahne]

Olaf Till writes:

> On Wed, Mar 01, 2017 at 02:06:29PM -0700, Brad Bell wrote:
>> On 03/01/2017 01:57 PM, Oliver Heimlich wrote:
>> ... snip ...
>> >I am not sure, if (and how easy) it is possibly to combine a
>> >templated(?) C++ library with types that are implemented in Octave as
>> >classes.
>> The Cppad Swig C++ library is an example of specializing a template library
>> to a particular type and then hooking that specialization up to a scripting
>> language.
>
> Swig makes an interface from Octave to C++. What Oliver would need is
> the opposite direction. This wouldn't work...
>
> I agree that passing an interval type, implemented with Octave
> classes, to an AD type would only be possible if AD is also
> implemented with Octave classes. This would need AD to be implemented
> first, and to make a specialization then which internally generates an
> interval type from the arguments (making it dependent on the interval
> package). But:
>
> 1. 'interval' must support vectors for this.
>
> 2. For AD itself to be useful, we need it to be thoroughly
>    implemented, including Hessians, at least.
>
> Olaf

Hi,

One way to implement automatic differentiation would be to do it with
Taylor arithmetic. For single variable functions this would allow for
computing arbitrarily order derivatives in an effective way. It can also
be generalized to multi-variable functions but I am not sure how much
work this is (though two variables should be no problem). This also fits
nicely with Olivers idea of integrating it with the interval
package. Taylor expansions are very suitable for evaluation with
integrals, one example of this can be found in [1]. Adding that
functionality to Octave would allow for very powerful interval methods.

I have created a proof-of-concept implementation of Taylor arithmetic
which is available at GitHub, [2]. So far it implements a constructor
and some basic operations, +, -, .*, ./, exp, log, .^, sin and cos. As
of yet there is no functions to actually make use of the result. It is
meant to work with the interval package and should then generate
guaranteed results.

I would be very interested in continuing this work as a
GSoC-student. Especially I am interested it developing it alongside the
interval package, it is in this environment I have used Taylor
arithmetic it before.

Regards,
Joel Dahne

[1] http://www2.math.uni-wuppertal.de/~xsc/preprints/prep_05_4.pdf
[2] https://github.com/Urathai/octave-taylor-POC