[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: hdf5

From: Daniel Heiserer
Subject: Re: hdf5
Date: Thu, 17 Aug 2000 09:03:07 +0200

>   Daniel> If hdf5 is not a database, is it hard to "combine" some databases
>   Daniel> like BerkeleyDB or GDB with hdf5 to create a scientific database?
> You don't say what you want to do with your data, whether it changes, whether
> you need subqueries etc

I just posted a message. I do not want todo subqueries, I want to use it 
to store all my data, which means, I have load, then change and then
the data again. Between two commands, I do not want to have nearly
anything in core. Of course it can be, that the matrix I store again
grows. And of course I want to delete files in the 'database'.
The reason I want the out of core capability is that I 
have to use many large matrices. 
A first step would be to load only the ones, which  I need for an
A second step should be to load only the part of a matrix I need 
for a function like a column or something like that.

> If you "merely" want to load and save lots of (static) matrices, I would
> conjecture that the builtin formats load and save will be hard to beat in
> terms of both speed and storage efficiency. You can easily build yourself
> small C, C++, Perl, ... programs that read/write in the same format as it
> "really" is just double array which you can store / retrieve with one
> command.
> If however you want to query "relations" between your data, then a relational
> database might be your choice.  I have used Octave with timeseries data on
> Sybase (expensive, proprietary) as well as PostgreSQL (free, BSD-style
> copyright).  I used Perl glue code (called from an .m file) back when I used
> Sybase, and I wrote two C++ .oct functions I posted here a long time ago
> which go directly to PostgreSQL.  Just as a reference, Bill Sharpe at
> Stanford (who is also a winner of an Economics Nobel price) has a little
> timeseries database system which is implemented direcly in Matlab and which
> might be portable.
> That being said, there are limits to what you can do due to design of the
> Octave (Matlab) language.  For full-blown data analysis, the S language is
> better, and I have very good recent experiences with the GNU R implementation
> of it and both the R interface to PostgreSQL as well as the ODBC interface.

Will the desing of Octave allow an efficient implementation 
of out of core handling of large (maybe sparse) matrices?


Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:
How to fund new projects:
Subscription information:

reply via email to

[Prev in Thread] Current Thread [Next in Thread]