[Octave-bug-tracker] [bug #56489] System freeze due to Excel import

[Philip Nienhuis]

Follow-up Comment #4, bug #56489 (project octave):

Right, that makes 64 MB for the numeric data alone if they were written as
doubles (but they are written as better-compressible text strings). Yeah I see
that you're hitting some resource limits.

To invoke the COM interface you'd need to also load the windows package and
have MS-Excel installed.
Have you ever checked the memory consumption of Excel when it has read the
file? Could be a lot more than Octave uses.

Another worthwile try would be:

## Open file pointer
xls = xlsopen ('v201981_rawdata.xlsx', 0);
## Read raw data
RAWARR1 = xls2oct (xls, 1);
## Close file pointer, along the lines release associated memory
xls = xlsclose (xls);
## Split data in numeric and text parts
[NUMARR1, TXTARR1, LIMITS1] = parsecell (RAWARR1);

(analogous to what xlsread itself does, but in another order).

A little warning: you'd get ~64 MB of doubles in NUMARR1, but a maybe tenfold
memory consumption for the cell arrays TXTARR1 and RAWARR1 (cell arrays have
an unbelievable RAM overhead not shown in "whos"), so those latter ones might
consume > 1 GB.

BTW it may be that your system just appears to freeze, but you can only be
sure if you look in Task Manager, Processes or Performance tab.
Given the size of your data I suspect your system may silently just be busy
with the regexps and may even be trashing (swapping RAM in and out from/to
virtual memory on disk). If the latter I'm not surprised that things appear to
grind to a halt.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?56489>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/