Re: algorithm for fminunc

[Jose]

On 01/18/2013 07:13 PM, Jordi Gutiérrez Hermoso wrote:
> I don't find the code too difficult to read. Upon further inspection,
> it is precisely damped BFSG with a dogleg trust region method. This is
> a fairly standard method. For example, you can read about it in
> Nocedal & Wright, section 18.4 (procedure 18.2). This is even using
> the same notation as the Octave source code. The dogleg line search is
> in op. cit. section 4.1.

Thanks for the reference, I found it.


> No other algorithm seems to be implemented here. Are you planning to
> add more algorithms? Does Matlab have an interface for choosing the
> algorithm? There could be improvements here.

According to
http://www.mathworks.se/help/optim/ug/fminunc.html
fminunc from Matlab can use two algorithms, one for large-scale 
optimization and another for medium-scale. BGFS is used in the second 
one. For the first one it uses an "interior-reflective Newton method" by 
Coleman et.al. The documentation provides the references to the 
appropriate articles.

I am not planning on implementing any new method, at least for now.


> Once I'm more certain that Nocedal & Wright is
> the appropriate reference, and it would help if you could confirm
> this, I will add the reference to the comments.

I'd love to help on this, but I am not an expert in optimization and it 
would take me too long at this moment to understand well what is going 
on under the hood.

But what about asking the original author? The code was written by 
Jaroslav Hajek, I hope he is reading this thread :)

Jose