Re: algorithm for fminunc

[Jordi Gutiérrez Hermoso]

On 18 January 2013 11:24, Jose <address@hidden> wrote:
> On 01/18/2013 03:48 PM, Jordi Gutiérrez Hermoso wrote:
>>
>> On 18 January 2013 07:29, Jose <address@hidden> wrote:
>>>
>>> I have tried to find what is the algorithm that fminunc uses for
>>> minimization, but I could not find a clear reference to it.
>>
>>
>> What about reading the source code? That is where the algorithm is. It
>> seems to be BFGS with dogleg line search.

> Yes, I did read through the source code trying to find a reference
> to the algorithm used. The only mention to BFGS was in line 189, in
> a comment that says "Use the damped BFGS formula". After that I
> reached the same conclusion than you: it seems to be BFGS.

I don't find the code too difficult to read. Upon further inspection,
it is precisely damped BFSG with a dogleg trust region method. This is
a fairly standard method. For example, you can read about it in
Nocedal & Wright, section 18.4 (procedure 18.2). This is even using
the same notation as the Octave source code. The dogleg line search is
in op. cit. section 4.1.

> But due to the "shyness" of the comment, the fact that Matlab can
> use different algorithms within the same function and that I am not
> so familiar with the method as to understand the code itself, I was
> asking for a confirmation.

No other algorithm seems to be implemented here. Are you planning to
add more algorithms? Does Matlab have an interface for choosing the
algorithm? There could be improvements here.

> IMHO it would be nice to have the algorithm used stated clearly in
> the documentation.

The guy who wrote this bit of code wasn't too fond of writing
long-winded comments. Once I'm more certain that Nocedal & Wright is
the appropriate reference, and it would help if you could confirm
this, I will add the reference to the comments.

- Jordi G. H.