Re: Matrix or array operations library

[Linas Vepstas]

Me too.

With one significant difference: I have extremely sparse arrays. Like, only
one-in-a-million array entries are non-zero.  And my arrays are hug -- say
2M by 2M, for a total of 4 tera-entries, of which only one in a million are
non-zero, so in fact, my data might fit in a gigabyte or less. (instead of
32 terabytes of RAM)

I solved this by writing my own ad-hoc library. The library user needs to
declare a half-dozen methods, indicating how to fetch the matrix entries,
what the dimensions of the array are, and methods to loop over the non-zero
row and column entries.  The library does the rest.

My code is quasi-special-purpose, and, well, just right for me, but a
serious dedicated library programmer could do a much better job than I
can.  My code is here:

https://github.com/opencog/atomspace/tree/master/opencog/matrix

The low-level API that the user must declare is here:
https://github.com/opencog/atomspace/blob/master/opencog/matrix/object-api.scm

Hmm, maybe I should pull this out of the project, and make it generic....

-- Linas


On Mon, Dec 24, 2018 at 4:01 PM Zelphir Kaltstahl <
address@hidden> wrote:

> Hello Guile Users,
>
> Is there some library, that enables high performance matrix operations
> or even n-dimensional array operations? I am thinking of something like
> NumPy in the Python ecosystem. I think NumPy in turn also uses some
> lower level thing to do what it does. I think OpenBLAS or MKL, depending
> on the architecture. I wonder if there is any wrapper around OpenBLAS
> for Guile or something similar.
>
> I am writing a little code for matrix operations and currently I am
> using Guile arrays, as they are made of vectors and have constant access
> time, which is already great. My guess is, that this would be the right
> choice if using pure Guile. I am writing data abstraction procedures, so
> that later on I could exchange what is used to represent the data.
>
> Maybe, if there is something like NumPy or lower level, I should use
> that instead? (Would I have to learn how to use FFI first?)
>
> Or maybe Guile's implementation is already so fast, that it would not
> make that much difference to use a lower level thing?
>
> Currently I have only a little experimental program, started today, so
> no huge plan. OK, one can fantasize about stuff like Pandas data frames
> in Guile, but I have no illusion, that it is a work of a few days or
> even weeks. It would be nice to learn, how to use a low level thing or
> maybe even Pandas, if there are any such bindings for Guile. I could
> make the implementation use different representations, depending on a
> parameter or something like that.
>
> Regards,
>
> Zelphir
>
>
>

-- 
cassette tapes - analog TV - film cameras - you