Re: Octave and Database connectivity

[Philip Nienhuis]

Jordi Gutiérrez Hermoso-2 wrote
> 
> On 12 September 2012 06:49, ahowe42 <ahowe42@> wrote:
>> For various reasons - including that I have much more expertise and
>> existing
>> matlab/octave code to use - I must stick with Octave.
> 
> I really recommend once more to use another language. Octave and
> Matlab are both essentially glorified wrappers over 30-year-old
> Fortran libraries and ill-suited for the programming needs of today.
> They are both awful general-purpose languages. Yes, people use these
> two languages to write web servers and email clients, but it's really
> a case of a square peg in a round hole. They do one thing and one
> thing oly well, and that is array-oriented numerical computation.
> Everything else about these two is unpleasant:
> 
>     http://abandonmatlab.wordpress.com/2009/07/23/hello-world/
> 

Maybe that's the natural inclination if you're a mathematician or software
engineer. But I see no reason to be that negative about Matlab and its OSS
siblings in other settings.

At work we do use Matlab (and Octave but I'm probably the only Octave user)
mainly for digging around in our hydrogeologic databases (in dBase, Excel,
.mat and text files and a colorful collection of legacy database programs),
sorting, filtering, and combining data, then we do some minor computation,
and then loads of visualization. In addition we use it for pre- and
postprocessing for groundwater models (usually GB volumes of text and binary
disk files per run). In short, we use it a sort of super-BASIC, but much
more versatile.
ML/Octave has proven to really be the easiest and best solution for those
purposes; in addition they are fast and -especially Matlab- deal with
visualization of big data volumes without a hitch. One can test and step
through the code anywhere, scripts and function files can be debugged any
moment, it allows for all our data manipulation and analyses to be scripted
thus be reproducible, and the basics are easily explained to our student
interns. Many function files in our code repository and archived projects
are over a decade and a half old and still work with no or minor mods.

So, while ML/Octave are superior interactive number crunchers, the
combination of that and ease of use as regards data I/O and graphics makes
for an extremely versatile multi-purpose tool.

One of my colleagues has tried Python as well but it turned into a bit of
disappointment (though I never asked what the exact problems were).



> :
> <snip>
>> I don't have any familiarity with database access through Python or
>> Java, but the Java implementation seems likely to be more efficient,
>> since I don't have to put a layer on top of Octave.
> 
> Oh, Java is an efficient layer? I suppose you have forgotten what it's
> like to wait for a jvm to boot up?
> 

Maybe .01 second or less?
And once the JVM is in memory, subsequent Java method calls are fast. Faster
than the ensuing SQL call.

Java itself is fast enough; but interfacing it from/to other universes
(Octave, Python, Mozilla, whatever) is slow of course. The trick would be to
write as much SQL functionality as possible in Java itself and let Octave
mainly do  I/O on preprocessed / filtered data (and then do number crunching
in Octave).



>  Ever wondered why LibreOffice takes
> so long to start? ;-)
> 

Because it is written in C++ perhaps?

Seriously, starting times for LibreOffice natively and through e.g.,
Octave/io pkg/Java differ negligibly.
It is initialization of LO & OOo that apparently takes a loooong time.



>> Does anyone have example code for running SQL queries against MySQL
>> using the Java package?
> 

I would be surprised if a bit of googling wouldn't turn up loads of hits on
this subject. Surely you aren't the first to need this functionality.

FWIW, I know that the database gurus at work have made a Java-based
Matlab-Oracle (-IIRC) connection to access the vast geoinfo / hydrological
databases from the water board. AFAIU the main motive was to avoid ML's
database toolbox levy for a network license.
By coincidence I'm in a meeting with them tomorrow so if there is time left
I could ask what they exactly made and how they did it.

As I wrote in another thread post yesterday, my experience until now is that
Octave's Java package seems less restrictive in the end than the commercial
competition's Java interface. ("in the end" as invoking Octave's Java
package has some (minor) syntax limitations) 
So with due respect to Jordi's opinion I'm not so pessimistic about invoking
Java for an SQL interface.



> There is a database package in OF, but I'm willing to bet it's
> suffering bitrot:
> 
>     http://octave.sourceforge.net/database/index.html
> 
> If you do get it to work, let us know.
> 

Yes it suffers bitrot. But maybe not so bad. It is on my list to attack once
I really need it (which may easily take a year or more). But as I learned
that a Java-based Matlab(/Octave?) - database bridge does work the priority
may not be so high now.

Philip




--
View this message in context: 
http://octave.1599824.n4.nabble.com/Octave-and-Database-connectivity-tp4643993p4644049.html
Sent from the Octave - General mailing list archive at Nabble.com.