
From:  Dmitrey Kroshko 
Subject:  Re: OpenOpt v 0.25  free optimization toolbox 
Date:  Sat, 16 Dec 2006 18:34:47 +0200 
Useragent:  Thunderbird 1.5.0.8 (Windows/20061025) 
Hi Michael, this is a nice idea but1) All OpenOpt solvers are very deeply integrated to the OpenOpt environment, so it's not a simple task to extract them and connect to Octave as quaziMATLAB routines fminsearch, fmincon etc (unfotrunately, OpenOpt currently doesn't have fmincon analog). Now almost all solvers are very simple, but if OpenOpt will implement full specter of constraints, 1st & 2nd derivatives, constraint & Hess patterns etc  it will be very hard every time to rewrote Octave functions as well (or do parallel implementations of changes  it will take much time & decriase speed of new ideas implementations; also, we are thinking about writing the same toolbox for SkiPy, which also will diminish the speed). 2) Even if I will wrote, for example, a routine similar to fmincon & call it fmincon  I must provide all fminconlike init parameters structure (that is equal to optimset(@fmincon) in MATLAB), and handle all these parameters (like TolPCG, MaxSQPIter, MaxPCGIter) correctly. Moreover, what if Mathworks will change fmincon calling syntax  rewrite its all again? Initially OpenOpt was intended as one half of my postgraduate work (now I have a year to complite this one) Then I decided to rewrote my code in TOMLABlike form, which is much more flexible than MATLAB. More about TOMLAB solvers & prices:
http://tomopt.com/tomlab/products/prices/commercial.phpI intended to propose TOMLAB our ralgbased software as one of their packages, but then I was convineced by icyb optim department workers that we should be opensourse followers. One of the key idea of OpenOpt  simultenious comparison of different (appropriate to the task) solvers, which is very well seen from OpenOpt graphic output. Currently I failed to plot in Octave even plot([1 2], [3 4]), but if you able to plot this  try comment line in OpenOpt/Kernel/ooInit, where I wrote "if exist('OCTAVE_VERSION'); prob.draw = @(x,p) []". Maybe you will got some graphical output. However, it took some minuties even wait for some text symbols to be printed in Octave, while MATLAB prints them regulary, I'm afraid the same will be with graphics output despite of my drawnow(). One more problem is that my output structure r is displayed very shortly in MATLAB and fully in Octave, that produces very much unnesesary output info. 3) Some algorithms show different effectiveness on different tasks, even grouped to the same class. for example, even for fmincon without Hesse info it could be proposed dozens of algorithms. Some are better on tasks with many nonlinear inequality constraints, some  with many equality constraints, some  with many constraints, but with only a few in optim point etc, etc. Automatic detection which one to use is very hard task, moreover, each year new optimization algorithms are appearing, new handling of penalties, of Lagrange multipliers etc. Handling them as fmincon parameters is very unconvinient, users probably will not even know that Octave fmincon has choise of lots handturning algorithms. 4) OpenOpt is intended to run as well in MATLAB  I guess, both MATLAB & Octave users will be wery glad to have such ability of their code with OpenOpt, and it will simplify their movement from MATLAB to Octave. 5) Currently further development of OpenOpt is under question. I will do this anyway at least one year  as long as it remains to be my postgraduate work. However, icyb optim department workers are not interested even in explaining some teoretical aspects to me (so in writing code for free too), because they are very busy with their own work; and currently I'm not skilled enough to study & implement latest algorithms (moreover, as a rule, written in English without pseudocode & precize algorithm description). BTW, currently along with studing I have to work in notMATLABrelated projects for to earn some money (80$/month). I don't know how much leading icyb optim department workers earn, I guess 1500...2500$ per month. BTW we have claster from World top 100 in our instityte with cputime prices much more less than in USA (and currently free for some icyb workers), & I intend to add parallel computations in OpenOpt. 6) current ooRun format is very usefull for running cell of problems on cell of solvers, i.e. ooRun(probs, solvers), which is absent in standart routines. It produses veryvell picture of convergence of different algorithms for the same task. Implementing this one as standart Octave feature will yield additional MATLABOctave incompability, btw in future Mathworks could inplement something like that, and problem of incompability wil raise ones again. 7) In future autodetection of appropriate solvers (according to the task involved) is intended. This feature is hard for implementing in MATLABlike optim routines style.
Now I asked savannah.gnu.org for any support, it will be nice if you (or any Octave developer) will wrote something positive in their discussion attached to my request
https://savannah.gnu.org/task/?6160 What about speed comparison with other Octave routines1) 1st problem is different implementation of cycles (for example, in annealing comparison), which are not handled by JITaccelerator or something like that; 2) 2nd problem is with different stopping criteria in OpenOpt & Octave , and their handturning. It will be very nice when I will got graphical output (plot()) working in Octave at last, currently it don't work. Then I will be able to construct something like example2.jpg from OpenOpt.zip and compare their speed. 3) 3rd problem is that my anneal & all other solvers are written in mfiles, and anneal is using forloop in a very bad way (for i=1:maxFunEvals vs 1:MaxIter of other solvers). So I guess it will fail speed comparison to your cfiles, moreover, without JIT accelerator. comparing via iteration number is implemented in OpenOpt, but iterations may be different even for the same algorithm (annealing) because of different inner start default option values; also remember that annealing is stochastic algorithm so it depends on random realization, so it must be run many times to compare.
WBR, Dmitrey.P.S. I didn't found my yestorday message to the helpoctave group, so let me announce link for OpenOpt downloading once again:
http://www.box.net/public/6bsuq765t4#main address@hidden пишет:
Hi, This looks pretty interesting. I'll have a look at it when I get some time, which unfortunately I don't have now. Did you know that there is a simulated annealing solver in octaveforge (samin)? It is written as an .oct function so that the loops are done in C. There is a samin_example program that shows how to use it. Also, there is a bfgs (and lbfgs) solver, also written as an .oct function, with an .m file frontend (bfgsmin.m). At some point, someone should try to do some benchmarking for the available solvers. It would be nice to select the ones that work best and that complement eachother, do some standardization of interfaces, and weed out the ones that are dominated by other alternatives. If you like what you see in samin and bfgsmin, please feel free to chop out and use anything that's useful. Regards, Michael
[Prev in Thread]  Current Thread  [Next in Thread] 