LuaJIT ffi issue with dynamic libraries

[Ekaitz Zarraga]

Hi all,

I've been having some issues with LuaJIT's FFI. More specifically with the
`load` function, that loads a library in the "search path for dynamic
libraries".

The LuaJIT documentation says the following:

> On POSIX systems, if the name contains no dot, the extension .so is appended.
> Also, the lib prefix is prepended if necessary. So ffi.load("z") looks for
> "libz.so" in the default shared library search path.


In my case I'm unable to load a library, in this case `librec.so` simply by
calling `ffi.load` even if both are part of the same environment. I read a
little bit of the LuaJIT code and I found that it is doing a `dlopen` inside so
I made a test:

First I made a simple C file with a call to `dlopen` and straced its result. It
simply finds the library after proving some folders. I also noted that the
environment automatically set the `LIBRARY_PATH` variable for me (even if the
value of the variable does not match the patch where librec.so is being
searched):

--8<---------------cut here---------------start------------->8---
$ guix shell gcc-toolchain strace recutils
$ cat <<EOF > dl.c
> #include<dlfcn.h>
>
> int main(int argc, char *argv[]){
>     void * i = dlopen("librec.so", RTLD_LAZY);
>     return (int)i;
> }
EOF
$ gcc dl.c -ldl
$ strace ./a.out 2>&1 | grep librec.so
openat(AT_FDCWD, 
"/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/28bcmy08ki5krvr2g9hbm3bhys822fvn-gcc-11.2.0-lib/lib/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/haswell/x86_64/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/haswell/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/x86_64/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/haswell/x86_64/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/haswell/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/x86_64/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/librec.so", 
O_RDONLY|O_CLOEXEC) = 3
$ echo $LIBRARY_PATH
/gnu/store/hiiyy2b8pw2cq9mw8iik9xi9lpclcv7b-profile/lib
--8<---------------cut here---------------end--------------->8---

When I do a similar approach with LuaJIT I get the following:

- The paths the `dlopen` call is proving are different to the previous case and
  they are pretty weird too.
- There's no `LIBRARY_PATH` set.
- Even if I set `LIBRARY_PATH` myself it's ignored.
- It works correctly when I set `LD_LIBRARY_PATH`, but should I?


--8<---------------cut here---------------start------------->8---
$ guix shell luajit strace recutils
$ cat <<EOF > dl.lua
> ffi = require "ffi"
> ffi.load "rec"
> EOF
$ strace luajit dl.lua 2>&1  | grep librec.so
openat(AT_FDCWD, 
"/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/x86_64/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/x86_64/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/x86_64/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../x86_64/librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../librec.so",
 O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, 
"/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
$ echo $LIBRARY_PATH

$ export LIBRARY_PATH=$GUIX_ENVIRONMENT/lib
$ strace luajit dl.lua 2>&1  | grep librec.so
#### SAME AS BEFORE ####
$ export LD_LIBRARY_PATH=$GUIX_ENVIRONMENT/lib
$ echo $LD_LIBRARY_PATH
/gnu/store/5v09kbdda7966x263cgnva2z15yrdbfb-profile/lib
$ strace luajit dl.lua 2>&1  | grep librec.so
openat(AT_FDCWD, 
"/gnu/store/5v09kbdda7966x263cgnva2z15yrdbfb-profile/lib/librec.so", 
O_RDONLY|O_CLOEXEC) = 3
--8<---------------cut here---------------end--------------->8---

Does any of this make sense?

The easiest conclusion is that LuaJIT is not working out the box and we should
fix it, but I'm not sure where to start because I'm assuming both examples here
come from the same concept: making a similar call to `dlopen`. There must be
some context I'm missing here to solve this.

Any ideas?


ElenQ Technology
Ethical Innovation