Re: Gettext: Suggested addition of memory allocation clean-up in'intl' library [Again]

[Svante Seleborg]

(snip)
> Svante Seleborg wrote:
> > > This is acceptable. The function should be called
> libintl_free_all().
> >
> > Good enough. If I implement such a function for *Win*32 only (since 
> > Ynix developers apparently only views things like clean-up as 
> > CPU-cycle-stealers), will you anyway consider merging it
> into the main
> > distribution
> 
> As I said in private mail already, whether I accept a patch in GNU 
> gettext, depends 1. on whether you convince me, 2.
> whether the patch is not platform dependent, easy to maintain and 
> similar.
> 
> About convincing me: It depends on the answer to question I asked:
> 
>   Why is this a problem? You can
>     1. reduce the amount of allocated memory by using LC_ALL=C,

It's not really about reducing the amount of memory. That is very low on the
list of priorities. Solid code is high on the list.
 
>     2. distinguish a memory leak from a static allocation by letting 
> your
>        program run a long time and then look at the memory amounts.

There are many ways, and many tools, to help writing solid and stable code.
There are also many principles of software engineering that have been learnt
the painful way by our common industry experience. Full control of memory
allocation is one of them - if you allocate something - ensure that you can,
and do, de-allocate. It's simply "good design".

As in all cases with "good design", you can always argue that it's not
necessary in "this specific case" because of such and such reason.

The biggest single argument against C/C++ is the difficulty of properly
managing heap-allocated memory. There are many other arguments, but they are
lesser in magnitude and popular impact. This has led to Java and C# to
mention two newer and popular platforms implementing garbage collection -
and still not really solving the problem. 

The truth of the matter is that just as Dijkstras "Go-to statement
considered harmful" led to the development of language constructs that limit
our choices of program control paths to avoid tangling ourselves in hopeless
messes of unstable code, we need similar controls on our use of heap memory.
As we're not supported by the language here, we must do so by ourselves.
Recall - there's no real need for if-then-else, do-while, case etc - we can
do this (and the compiler of course does for us) with the good old goto. As
long as we limit our use of the feature to a set of controlled methods, we
are "safe".

One of those priniciples for using heap-allocated memory is: "If you have
code to allocate it, you must include code to release it". It's as simple as
that.

Saying "it'll get released when the program exits anyway" is _not_
sufficient. The program may _never_ exit - and we're talking about a library
which we have no knowledge before-hand at all what it'll be used for.

So my argument to convince you about this is based on that simple statement.
"You allocate, thus you must free."

> 
> > apparently you as maintainer do maintain that the feature is 
> > unnecessary, and even harmful, anyway in a Unix environment
> 
> That's a misunderstanding. When I ask you to show that the feature is 
> necessary, it's not because I think it's unnecessary. It's rather to 
> have you state concretely what is the goal of the feature, so that we 
> can see whether this feature or another one is better for reaching the 
> goal.

The goal of the feature in this question is to allow a program incorporating
the library to follow a golden rule of using heap-allocated memory, and not
have to depend on such momentuous events as program termination (how do you
know the program will/can _ever_ terminate? How do you know that the program
has resources to keep texts in allocated memory but for the briefest of
times?).

The goal of the feature is to enable the writing of solid, stable software,
that has 100% control of heap-allocated memory.

> 
> > for us poor *Win*32-developers who do need such crutches - and are 
> > willing to pay the price in CPU-cycles?
> 
> Who said that atexit(free_all_memory_blocks()) would not be costly on 
> Woe32? It will be costly
>   1) on all systems that implement virtual memory with swapping of
>      pages, and Woe32 does so,
>   2) on all systems that have an L1 or L2 memory cache - and most
>      CPUs nowadays have.

I'm afraid you misread my statement. I did not claim that
atexit(free_all_memory_blocks()) would be for free in *Win*32. I said at
least I'm willing to pay the price! Just as I'm willing to pay the price for
parameter validation, buffer length checks, various assert conditions,
structured programming, compilers, shared libraries, code re-use,
abstraction, inheritance and all the other mechanisms used to be productive
and produce solid code. All of these will cost execution time compared to a
monolithic no-holds-barred implementation in assembler. But we don't do that
- do we? Because we're willing to pay the price.

But the currently proposed feature, which is to implement libintl_free_all()
as a voluntary call, suffices since the application programmer can decide
when/if to call it - including in an atexit() call.

> 
> > I do want to
> > implement the feature, and lack the time to test in anything but a 
> > Windows enviroment
> 
> If you can send a patch for a platform-independent feature, better 
> avoid platform-specific #ifdefs. The fact that you could test it one 
> platform only is worth mentioning when you submit it - so that the 
> maintainer knows what to test on -, but is not a reason for making the 
> patch platform-specific.

I was hoping to _remove_ a few of the approximately 540 #if's that are now
part of just the 'intl' base-library. The problem with it is that you may
not agree, since a portable implementation will probably cost a few extra
CPU-cycles, even when not used since we need to keep track of allocated
memory - which code is #ifdef:ed away for *Win32* in the current release.

I also asked for documentation for the libc_freeres_ptr and libc_freeres_fn
macros - you pointed to the _definition_ in header files, one which begins
with:

/* This file contains a number of internal prototype declarations that
   don't fit anywhere else.  */

I note three things with the useage of libc_freeres_ptr etc.

1 - The word "internal" together with the fact that
2 - they are not documented in the libc manual.
3 - practically no-one uses it. (402 hits on Google - most with gettext or
libc itself...)

My question based on this: Should gettext really be using these? I'm very
wary of using undocumented features. Having access to the source code is no
excuse. Internal interfaces and functions are just that - software depending
on semantics of internal functions deduced from reading the source code are
dangerous.

There's also the problem that you do not know which malloc() will be called
- it's perfectly ok and in some cases necessary to provide a private
implementation. If the libc_freeres_X macros assume that it knows something
about which malloc() is used, we might break the program when we implement a
private heap allocation implementation. From the definition in the libc
internal header files, it's not exactly trivial to figure out what
libc_freeres_X does, so I can't really tell if this is a problem or not -
but the point is that it _might_ be, or become in the future.

So, what I'd like to do, is to implement libintl_free_all() in the
main-stream code, with no #ifdefs, and no dependency on libc_freeres_X
macros etc.

Comments please.

Svante