Re: [External] : Passing buffers to function in elisp

[Stefan Monnier]

> Also, if I interpreted profiler's hieroglyphs correctly, it told me that
> this setq
>
>   (setq stream (vconcat stream (plist-get page :stream)))

This is a typical a source of unnecessary O(N²) complexity: the above
line takes O(N) time, so if you do it O(N) times, you got your
N² blowup.  You're usually better off doing

    (push (plist-get page :stream) stream-chunks)

and then at the end get the `stream` with

    (mapconcat #'identity (nreverse stream-chunks) nil)
or
    (apply #'vconcat (nreverse stream-chunks))

Of course that depends on what else happens with `stream` (I haven't
really looked at your code, sorry).

> I think I can replace vectors with strings, which should, according to
> the elisp manual, "occupy one-fourth the space of a vector of the same
> elements."

More likely one-eighth nowadays (64 bit machines).

> Similarly bindat consumes a lot of memory.

Hmm... IIRC it should not use up very much "auxiliary" memory.
IOW its memory usage should be determined by the amount of data it returns.
So, when producing the bytestring it should be quite efficient memorywise.
When reading the bytestring it may be wastefully allocating memory for
all the alists (and also it may be wasteful if you only need some info
because you still need to parse everything and allocate data to
represent its parsed form).

> But bindat internals are beyond me.

I can be of help here :-)

>> That's definitely something to consider.  Another is whether the ELisp
>> code was byte-compiled (if not, then all bets are off, the interpreter
>> itself generates a fair bit of garbage, especially if you use a lot of
>> macros).
> No, it was not byte-compiled.

Then stop right there and fix this problem.  There's absolutely no point
worrying about performance (including memory use) if the code is
not compiled because compilation can change the behavior drastically.

The only reason to run interpreted code nowadays is when you're
Edebugging a piece of code.

> I'll try byte-compiling after the code is in good enough shape to do
> controlled experiments.

The compiler is your friend.  He can help you get the code in good shape :-)


        Stefan