help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: File size limits in textread()?


From: Paul and Sandra Koufalas
Subject: Re: File size limits in textread()?
Date: Tue, 14 Jun 2005 17:08:54 +0930
User-agent: Mozilla Thunderbird 0.8 (Windows/20040913)

Thanks Stefan,

I'll give that code a go.

I had success with Keith and Dmitri's suggestions. There was an earlier post on textread() that John E replied too, with a patch for textread(), and I manually incorporated those changes too. Initialising all of the varargout data structures made a huge difference, order of magnitude at least. Took about 20s to load that 6000 row, 10 column table.

I also discovered csv2cell(), and being dynamically linked code it's very fast, but I don't know how to convert cell arrays with strings to plain old vanilla string arrays. For columns with numerical data, using [mycellarrayname{:,columnnumber}]' worked fine. I assign the result to a structure variable as a field. But with strings it resulted in each character of the string going on seperate rows. I'm sure there's a way and it's probably simple!

Cheers,
Paul.

Stefan van der Walt wrote:

Hi Keith

On Fri, Jun 10, 2005 at 10:20:59AM -0700, Keith Goodman wrote:
On 6/10/05, Stefan van der Walt <address@hidden> wrote:
Here's a much faster version of the code posted earlier.
Wow. It is fast. Can someone change the output of tread to be a matrix
for numbers instead of a cell (see below)?

That shouldn't be too hard.  One must just modify the function
set_cell_data to accept two template arguments in stead of one: a
container_type and a column_type.

The only thing that might be tricky is keeping track of all the output
matrices.  I thought I could do this using an octave_value_list, but
it's not so easy to get to the actual representation of an octave_value.

E.g., in the case of a matrix, we need to reach the octave_value's
internal matrix representation (without making any copies of the
matrix in memory).

Alternatively, we can keep a list of void pointers, but that seems
like the dirty (dangerous) way of doing things.

If anyone knows how to handle this problem, I'll be glad to know.
("This problem" referring to keeping a list of differently typed
objects, while retaining the ability to access the objects).

Regards
Stefan



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------




-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------



reply via email to

[Prev in Thread] Current Thread [Next in Thread]