ltdl bugs

[Jeff Squyres]

I think that I have found 2 bugs in libltdl (1.5.14).  These are 
repeatable on Linux 2.4 machines:

1. If you lt_dlopen(NULL) and then lt_dlclose() the handle that was 
returned, a memory leak ensues.  This is easily reproducible by writing 
a short program:

-----
#include <ltdl.h>

int main(int argc, char* argv[]) {
    lt_dlhandle handle;

    lt_dlinit();
    handle = lt_dlopen(NULL);
    lt_dlclose(handle);
    lt_dlexit();
    return 0;
}
-----

and then run that application through valgrind with memory leak 
checking enabled (e.g., valgrind --num-callers=100 --tool=memcheck 
--leak-check=yes --show-reachable=yes).  Here's a snipit showing the 
leak:

-----
==20985== 44 bytes in 1 blocks are definitely lost in loss record 1 of 1
==20985==    at 0x1B905EDD: malloc (vg_replace_malloc.c:131)
==20985==    by 0x8048F38: lt_emalloc (ltdl.c:1019)
==20985==    by 0x804A84A: try_dlopen (ltdl.c:3061)
==20985==    by 0x804B3C6: lt_dlopen (ltdl.c:3423)
==20985==    by 0x8048BF4: main (lttest.c:8)
-----

The problem appears to be on ltdl.c:3801, in lt_dlclose():

  if (handle->info.ref_count <= 0 && !LT_DLIS_RESIDENT (handle))
    /* followed by everything to unload the module and free
       the associated data and handle */

The ref_count *is* 0, but !LT_DLIS_RESIDENT(handle) is false (because 
handle->flags is 1).  Therefore, ltdl decides not to actually do the 
freeing -- and the handle is leaked.

I'm not sure what the intent of the DLIS_RESIDENT test is for, but I'm 
guessing that it's to ensure that you don't actually try to unload the 
program itself (for the case where you lt_dlopen(NULL)).  If this is 
true, then the problem is solved by splitting this logic up a little, 
something like:

  if (handle->info.ref_count <= 0) {
    if (!LT_DLIS_RESIDENT (handle)) {
       /* stuff to unload the module */
    }
    /* stuff to free the data and handle */
  }

I haven't fully audited what is happening, but I think that this might 
be the correct solution...?

2. We're getting warnings from "gcc -pedantic" when we use lt_dlsym() 
to look up function pointers, because there is no legal casting from 
(void*) to a function pointer (because some architectures allow them to 
be different sizes).  I think that this has been documented elsewhere 
(of course, I can't find it via google now...).  For example:

-----
#include <ltdl.h>

typedef int (*fn_t)(int, char **);

int main(int argc, char* argv[]) {
    lt_dlhandle handle;
    fn_t ptr;

    lt_dlinit();
    handle = lt_dlopen(NULL);
    ptr = lt_dlsym(handle, "main");
    lt_dlclose(handle);
    lt_dlexit();
    return 0;
}
-----

Compiling with -pedantic gives:

-----
lttest.c:12: warning: ISO C forbids assignment between function pointer 
and `void *'
-----

Can a second function be added, perhaps lt_dlsymfunc(), that returns 
function pointers instead of (void*)?

Many thanks.

--
{+} Jeff Squyres
{+} address@hidden
{+} http://www.lam-mpi.org/