[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: OpenOpt v 0.25 - free optimization toolbox

From: Dmitrey Kroshko
Subject: Re: OpenOpt v 0.25 - free optimization toolbox
Date: Sat, 16 Dec 2006 18:34:47 +0200
User-agent: Thunderbird (Windows/20061025)

Hi Michael,
this is a nice idea but
1) All OpenOpt solvers are very deeply integrated to the OpenOpt environment, so it's not a simple task to extract them and connect to Octave as quazi-MATLAB routines fminsearch, fmincon etc (unfotrunately, OpenOpt currently doesn't have fmincon analog). Now almost all solvers are very simple, but if OpenOpt will implement full specter of constraints, 1st & 2nd derivatives, constraint & Hess patterns etc - it will be very hard every time to rewrote Octave functions as well (or do parallel implementations of changes - it will take much time & decriase speed of new ideas implementations; also, we are thinking about writing the same toolbox for SkiPy, which also will diminish the speed). 2) Even if I will wrote, for example, a routine similar to fmincon & call it fmincon - I must provide all fmincon-like init parameters structure (that is equal to optimset(@fmincon) in MATLAB), and handle all these parameters (like TolPCG, MaxSQPIter, MaxPCGIter) correctly. Moreover, what if Mathworks will change fmincon calling syntax - rewrite its all again? Initially OpenOpt was intended as one half of my post-graduate work (now I have a year to complite this one) Then I decided to rewrote my code in TOMLAB-like form, which is much more flexible than MATLAB. More about TOMLAB solvers & prices:
I intended to propose TOMLAB our ralg-based software as one of their packages, but then I was convineced by icyb optim department workers that we should be opensourse followers. One of the key idea of OpenOpt - simultenious comparison of different (appropriate to the task) solvers, which is very well seen from OpenOpt graphic output. Currently I failed to plot in Octave even plot([1 2], [3 4]), but if you able to plot this - try comment line in OpenOpt/Kernel/ooInit, where I wrote "if exist('OCTAVE_VERSION'); prob.draw = @(x,p) []". Maybe you will got some graphical output. However, it took some minuties even wait for some text symbols to be printed in Octave, while MATLAB prints them regulary, I'm afraid the same will be with graphics output despite of my drawnow(). One more problem is that my output structure r is displayed very shortly in MATLAB and fully in Octave, that produces very much unnesesary output info. 3) Some algorithms show different effectiveness on different tasks, even grouped to the same class. for example, even for fmincon without Hesse info it could be proposed dozens of algorithms. Some are better on tasks with many non-linear inequality constraints, some - with many equality constraints, some - with many constraints, but with only a few in optim point etc, etc. Automatic detection which one to use is very hard task, moreover, each year new optimization algorithms are appearing, new handling of penalties, of Lagrange multipliers etc. Handling them as fmincon parameters is very unconvinient, users probably will not even know that Octave fmincon has choise of lots hand-turning algorithms. 4) OpenOpt is intended to run as well in MATLAB - I guess, both MATLAB & Octave users will be wery glad to have such ability of their code with OpenOpt, and it will simplify their movement from MATLAB to Octave. 5) Currently further development of OpenOpt is under question. I will do this anyway at least one year - as long as it remains to be my post-graduate work. However, icyb optim department workers are not interested even in explaining some teoretical aspects to me (so in writing code for free too), because they are very busy with their own work; and currently I'm not skilled enough to study & implement latest algorithms (moreover, as a rule, written in English without pseudocode & precize algorithm description). BTW, currently along with studing I have to work in not-MATLAB-related projects for to earn some money (80$/month). I don't know how much leading icyb optim department workers earn, I guess 1500...2500$ per month. BTW we have claster from World top 100 in our instityte with cputime prices much more less than in USA (and currently free for some icyb workers), & I intend to add parallel computations in OpenOpt. 6) current ooRun format is very usefull for running cell of problems on cell of solvers, i.e. ooRun(probs, solvers), which is absent in standart routines. It produses very-vell picture of convergence of different algorithms for the same task. Implementing this one as standart Octave feature will yield additional MATLAB-Octave incompability, btw in future Mathworks could inplement something like that, and problem of incompability wil raise ones again. 7) In future autodetection of appropriate solvers (according to the task involved) is intended. This feature is hard for implementing in MATLAB-like optim routines style.

Now I asked for any support, it will be nice if you (or any Octave developer) will wrote something positive in their discussion attached to my request
What about speed comparison with other Octave routines-
1) 1st problem is different implementation of cycles (for example, in annealing comparison), which are not handled by JIT-accelerator or something like that; 2) 2nd problem is with different stopping criteria in OpenOpt & Octave , and their hand-turning. It will be very nice when I will got graphical output (plot()) working in Octave at last, currently it don't work. Then I will be able to construct something like example2.jpg from and compare their speed. 3) 3rd problem is that my anneal & all other solvers are written in m-files, and anneal is using for-loop in a very bad way (for i=1:maxFunEvals vs 1:MaxIter of other solvers). So I guess it will fail speed comparison to your c-files, moreover, without JIT accelerator. comparing via iteration number is implemented in OpenOpt, but iterations may be different even for the same algorithm (annealing) because of different inner start default option values; also remember that annealing is stochastic algorithm so it depends on random realization, so it must be run many times to compare.
WBR, Dmitrey.

P.S. I didn't found my yestorday message to the help-octave group, so let me announce link for OpenOpt downloading once again:

address@hidden пишет:
This looks pretty interesting. I'll have a look at it when I get some
time, which unfortunately I don't have now. Did you know that there is
a simulated annealing solver in octave-forge (samin)? It is written as
an .oct function so that the loops are done in C. There is a
samin_example program that shows how to use it. Also, there is a bfgs
(and l-bfgs) solver, also written as an .oct function, with an .m file
frontend (bfgsmin.m). At some point, someone should try to do some
benchmarking for the available solvers. It would be nice to select the
ones that work best and that complement eachother, do some
standardization of interfaces, and weed out the ones that are
dominated by other alternatives. If you like what you see in samin and
bfgsmin, please feel free to chop out  and use anything that's useful.
Regards, Michael

reply via email to

[Prev in Thread] Current Thread [Next in Thread]