octave && database

[Daniel Heiserer]

Hi,

has ever anybody thought of coupling octave with a free database,
which takes care of octaves data (e.g. postgres).

Currently octave keeps all in core, which is nice for
small problems and big machines. I have here problems
where I deal with really big matrices 1e6*1e6 (sparse of course)
and bigger.
I also have full matrices, which have the size of many GB.
This is nothing what octave could handle with it's current
data-structure.
If you look at software packages, which can handle such
data, without having 5GB of main memory then you see a smart 
approach, which may be not perfect for an interactive tool like octave.
(?)
They have a database where they store everything. Nothing
is kept in core execept precompiled functions and 
1x1 "parameters". For a multiplication of a
A*B matrix they can get very efficently 
1 row from matrix A and 1 column from matrix B from the
database multiply it and store the result on the database,
then they take the next one. The maximum amount of core
memory you need is in the order of 
"max(length(column),length(row))".

If you look at the interactivity which is very important for a tool like
octave, all the user really needs
is to review his data in a :
o       a plot 
o       on the screen.
o       ???
and that as fast as possible.
If you have a fast database, which can support direct
access to what you need I think that would be a fast 
enough approach for interactivity.

All the parsing stuff and language could remain.
Maybe also new data structures could be integrated easily using
a object-oriented database as a backend.

What do others think about that?
Would octave be capable of that, or would that mean a *TOTAL* 
new software.

daniel



---------------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.  To ensure
that development continues, see www.che.wisc.edu/octave/giftform.html
Instructions for unsubscribing: www.che.wisc.edu/octave/archive.html
---------------------------------------------------------------------