Re: Performance optimization (allocation inside a for loop)

[Przemek Klosowski]

   Is it possible to adjust the Octave's allocation algorithm so that it
   could allocate larger chunks of data (or growing chunks of data)?

The common way of doing what you're asking is to over-allocate,
e.g. on filling up the current size-N block, re-allocate a new block of
size 2N; this way the allocation and copying cost is proportional to
the log(size) rather than size of the array.

The problem for Octave is that it results in wasting a constant
fraction of the total allocated size, which is a critical no-no for
anyone with large data sets.

I wonder if the allocation algorithm could be modified to
over-allocate a constant _amount_ (rather than constant fraction)---or
maybe a constant fraction up to certain array size and constant amount
above that. Of course this approach would require arrays to have two
sizes: user-visible and internal---because we don't want the
over-allocation to be shown in length(array)