Re: Speed of odsread

[siko1056]

hjborsje wrote
> I'm comparing the speed of loading a LibreOffice (*.ods) spreadsheet in
> Octave and MATLAB.  The results are so vastly different that I wonder if
> I'm
> doing it correctly.  A spreadsheet with 1900 rows, 23 columns takes 135
> seconds in Octave. In Matlab 5.5 seconds.   A larger spreadsheet of 60,000
> x
> 23 takes 10.5 seconds in Matlab. Octave did not finish after 10 minutes.  
> I
> run Octave 4.2.1 with all the latest packages on Windows 10.  I enter:
> [s,~,~,~] = odsload ('MySpreadsheet.ods',1);  Adding a third and fourth
> parameters makes no significant difference. RData_01b.ods
> <http://octave.1599824.n4.nabble.com/file/t372538/RData_01b.ods>  

I assume you are using this code [1]? Java is not as native to Octave as it
is to Matlab (the whole GUI is Java), so any attempt to use Java is
overhead. I tried  the odsread function from the io package [2]. But the
results are not promising as well. To use, just type:

>> pkg install -forge io
>> pkg load io
>> javaaddpath ("jOpenDocument-1.3.jar")          # Available from [3]
>> javaaddpath ("xerces-2_11_0/xercesImpl.jar") # Available from [4]

>> N = 10; t = 0; for i = 1:N, tic; X = odsread ('RData_01b.ods'); t = t +
>> toc; end, fprintf('csvread: avg. %f seconds\n', t/N)

csvread: avg. 10.013385 seconds

It is a bit time saving, when having multiple reads to open the document
once and only invoke the reading, but not that much better:

>> ods = odsopen ('RData_01b.ods'); N = 10; t = 0; for i = 1:N, tic; X =
>> ods2oct (ods); t = t + toc; end, fprintf('csvread: avg. %f seconds\n',
>> t/N), ods = odsclose (ods);

csvread: avg. 7.611418 seconds


Anyway, when dealing with a huge amount of data, I think that an Excel- or
ODS-spreadsheet is like running a marathon in   knight's armor, it is an
unnecessary overhead. Simply save your data from your application of choice
as comma separated values (CSV) and see the magic that works even far beyond
60,000 data rows:

>> N = 10; t = 0; for i = 1:N, tic; X = csvread ('RData_01b.csv'); t = t +
>> toc; end, fprintf('csvread: avg. %f seconds\n', t/N)

Matlab R2017a: csvread: avg. 0.014881 seconds
Octave 4.2.1: csvread: avg. 0.020552 seconds

Kai.

[1]:
https://www.mathworks.com/matlabcentral/fileexchange/28411-read-and-write-open-document-format--odf--spreadsheet---ods-
[2]: https://octave.sourceforge.io/io/function/odsread.html
[3]: http://www.jopendocument.org/downloads.html
[4]: https://xerces.apache.org/mirrors.cgi#binary

RData_01b.csv
<http://octave.1599824.n4.nabble.com/file/t370282/RData_01b.csv>  



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html