help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

loading big gzip compressed and uncompressed numerical matrices


From: Andreas Weber
Subject: loading big gzip compressed and uncompressed numerical matrices
Date: Sat, 21 Dec 2019 15:20:37 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.3.0

Dear all,

I've implemented a CSV parser as OCT file [1] to load gzip compressed
numerical matrices.

The reason for this was that I'm running GNU Octave on embedded systems
and I had to deal with increasing, gzip compressed files (see my
question 3 weeks ago [2]).

The result is quite surprising (at least for me): It's approx. 12 times
faster than the builtin "load" even for non-incremental reads.

I hope it can be useful for others who deal with large numerical CSVs.

-- Andy

[1] https://github.com/Andy1978/load_gz/tree/master
[2]
https://octave.1599824.n4.nabble.com/incremental-read-of-gzipped-matrix-td4694821.html





reply via email to

[Prev in Thread] Current Thread [Next in Thread]