Re: [lwip-devel] mem(p) allocator change request

[Kaos]

address@hidden wrote:

[snip]

> The thing is, no-one has actually shown that memory fragmentation is
> actually an issue for the allocation patterns we have in lwIP. Basically,
> we allocate lots of small, similar sized objects, which mostly have short
> lifespans.

The issue will arise when combined with the rest of the application, 
which has its own pattern of allocations, which might result in major 
fragmentation, simply because of the pattern from lwIP.

> The reason people are concerned about fragmentation is that it could
> result in unused (and unusable) bits of memory stuck between two
> long-lived objects. First, this is unlikely to happen in lwIP, and second,
> the current solution solves the problem by wasting even more space ---
> that is, unless you know exactly how many of each pool type you will need
> in advance --- which I doubt (m)any people do.

For my current application, fragmentation might not be a big problem in 
this case, but I prefer to be on the safe side for a stable application.
Not that much memory is wasted, by having mem stats running along with 
the app, mem usage is logged and can be tuned according to demand.
Using a set of block-pools with commonly used block sizes and a general 
byte-pool for arbitrary large (and quite small, short life blocks) 
mapped through a general alloc routine makes it quite usefull.

> The memp solution is just to statically allocate a number of buffers of
> each type, which means you invariably waste a significant amount of memory
> on buffers that are never used. Switching to a single, general purpose
> allocator removes the waste of this memory, as it can be used for other
> types when needed.

The general purpose argument holds for a memp solution as well without 
significant amount of memory going to waste (as argued above).

> Now we can all hand-wave about fragmentation, but until someone
> demonstrates that it is an issue in practice, and that more memory is lost
> to fragmentation than is wasted by using a static number of buffers, we
> should not assume the memp implementation to be more space efficient.

It's stability in the long run I'm after, not the most space efficient 
allocator.

>
[snip]
> I'm also happy to work on this once the new pbuf implementation is merged,
> but I think the best design would be to define a struct containing a bunch
> of function pointers to malloc / realloc / free (and possibly init /
> deinit - which would help solve current cleanup problems in lwIP). The
> user could then register one or more of these structs to be used at
> different points in lwIP --- thus allowing an arbitrary number of
> allocators, which can be as specialised as you like, or alternately you
> can pass a single general purpose allocator to everything --- it is, as it
> should be, the developers choice.

Agreed :c)
Not sure how important it is to be able to have different allocators for 
different modules, though.

// Andreas