Re: [bug-recutils] Optimizing memory consumption

[Jose E. Marchesi]

    > There are no memory leaks.  As you can see, the peak indicates a total
    > of memory used of around 121Mb in the heap.  This is simply too much for
    > a 4 megabytes recfile, so we must come with some optimizations in terms
    > of memory consuption.
    
    There are some leaks with othoer options (at least with some indexing
    changes), planning to find and fix more after the index support is
    complete.  I don't expect this to have significant effects.

Running the torture tests with memcheck will probably expose the leaks.
But you are right: the big problem is the memory consumption not the
leaks.
    
    > For example, an obvious optimization is to use a pool of strings to
    > store the field names.  The pool would be shared by all the records
    > pertaining to some record set.
    
    Field sources might be more commonly duplicated.  Similarly for values
    of some fields.

I am covering both field sources and field names in a private branch
that I will commit soon.  It uses a new ADT called spool (for strings
pool).
    
    Seen many places in librec doing deep copies of objects, this would
    probably need reference counting or a more complex solution to use
    references to the same objects instead.

For example?

-- 
Jose E. Marchesi         http://www.jemarch.net
GNU Project              http://www.gnu.org