Re: octave and image processing

[Paul Kienzle]

You could define a matrix type which leaves the matrix on the disk and only
loads the parts of it that are needed.  You will need to write a number of
functions to operate on this file in a block-by-block manor, but the
algorithms for doing so are similar to the algorithms required for parallel
processing, so most of what you need has already been developed.  As an
added bonus, you can feed your problems to an smp or numa system and your
programs will run faster too.

Paul Kienzle
address@hidden

----- Original Message -----
From: "Al Niessner" <address@hidden>
To: <address@hidden>
Sent: Thursday, December 19, 2002 7:58 PM
Subject: octave and image processing


>
> How can/should octave be changed to allow for processing very large
> images?
>
> In case you have not guessed by my sudden increase in traffic on this
> list, I am looking at octave to replace some home grown math engine use
> to process very large images -- several gigabytes per image. The current
> tool uses Fortran at its heart and has Fortran grammar and passes
> everything by reference. This in turn reduces the memory consumption
> which is important considering the ubiquity of 32 addressing machines.
>
> The first thing to test was converting our C/C++ and Fortran algorithms
> to use with octave. Piece of cake.
>
> The second thing to determine is how to minimize the memory
> requirements. Although the cost of memory is cheap, 32 bit addressing is
> ubiquitous right now; thus, the 4 GB limit is too. This is a real
> problem because I would like to load images close to that limit. To make
> life simple, lets say a 16k x 16k image which should be 2 GB. One
> function call and I am out of memory because of the 32 bit addressing.
>
> I have been trying to think how to allow for very large images inside of
> octave. I should note that I am not willing to make large changes to
> octave in order to do this because it would then have all the same
> problems as the current tool -- maintainability. So I have come up with
> two possible solutions: One, create a user defined type called 'image'
> that would behave differently than a matrix in that it would never be
> copied. Two, create a user defined type call 'image_handle' that would
> allow access to the image data in the way a file handle allows access to
> files. By themselves, I do not think either of these will work in the
> end.
>
> I then thought that a 'subimage' would be necessary in order to allow
> the rest of octave to treat it as a true matrix. In this way, one could
> still operate on portions of the image -- much smaller than 2 GB -- as a
> true matrix and then use dynamically loaded functions (.oct files) that
> could do in-place operations do full image manipulation. If that were
> the case, I would make 'image' directly immutable to the interpreter,
> but allow .oct files to manipulate it in-place since passing by
> reference is more natural to C/C++ and Fortran than the octave compiler.
> The hurdle in this instance is do you make the programmer reintegrate
> the subimage or should it be done automatically. Of course, there could
> be two types of subimages where the caller could choose at creation time
> if they want implicit or explicit reintegration of the subimage into the
> original image.
>
> These are the best ideas I could come up with given my limited
> familiarity with octave. Since many of you have a lot more experience, I
> would appreciate any and all guidance you can give me.
>
> Al Niessner
>
>
>
>
> -------------------------------------------------------------
> Octave is freely available under the terms of the GNU GPL.
>
> Octave's home on the web:  http://www.octave.org
> How to fund new projects:  http://www.octave.org/funding.html
> Subscription information:  http://www.octave.org/archive.html
> -------------------------------------------------------------
>



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------