Re: static build

[Alex Zimnitsky]

That was quite an answer, a rare thing nowadays.
Many thanks for your time!


Well, I built coreutils in a tweaked freebsd environment
with gcc (8.3.0) and clang (7.0.1).

my build commands were like this:

export LDFLAGS='-static'
./configure --enable-threads=posix --disable-silent-rules\
            --enable-single-binary=symlinks
make

The first problem encountered was failure to detect libiconv and gettext
when -static is used. The configure script prefers /usr/lib/libiconv.so
and /usr/lib/libintl.so (and their .a counterparts) to -liconv and
-lintl during tests, resulting in linking -static against a shared
object.   It still uses -lgmp, though.

Had to move libiconv.so and libintl.so out of the way and restore them
afterwards to build coreutils with gettext support.

The second issue appeared only when building with clang. For some 
reason it still considers the system suitable for stdbuf when -static is
given. Had to add 
           
--enable-no-install-program=arch,coreutils,hostname,stdbuf

to configure.

Finally, everything was built statically.

I also compared two static builds with and without
--enable-single-binary
which resulted in ~8MB vs. ~250MB

I'll sure stick with --enable-single-binary!


cheers,

Alex


On Sat, 2019-04-06 at 22:33 -0600, Assaf Gordon wrote:
> Hello,
> 
> On 2019-04-06 9:15 a.m., Alex Zimnitsky wrote:
> > I would really appreciate if coreutils could be built statically
> > with something like bash's --enable-static-link parameter co configure.
> 
> There is no single switch like bash's, but I can offer few methods
> to build static coreutils binaries.
> 
> First,
> I assume you are trying to build on a GNU/Linux system,
> with glibc and gcc.
> That is because in other configurations (non-glibc or non-linux),
> there are actually less complications (due to less detected features),
> and this typically "just works":
> 
>       ./configure LDFLAGS="-static"
>       make
> 
> 
> Second,
> Understand that on most modern GNU/Linux systems (using dynalic glibc),
> static building is tricky depending on which functions you use.
> If your program use getpwnam(3) and similar functions, they will still
> require the dynamic glibc during run time (due to the NSS mechanism).
> 
> Older Debians and Redhats provided a "glibc-static" package,
> which allowed static binaries easily, but this is not available
> any longer.
> 
> The result is that few coreutils programs will run at reduced
> features when built statically, and executed on a different system.
> Examples:
>    "id" and "groups" will print numeric UID/GID, but not names.
>    "chown" will accept numeric UID/GID, but will fail when given names.
> 
> 
> 
> Third,
> The "coreutils" package includes a program named stdbuf(1) that
> on linux with GCC builds a shared library as a dependency.
> Building this library fails with static builds, but there
> is currently no user-friendly way to disable it.
> https://lists.gnu.org/archive/html/coreutils/2010-07/msg00010.html
> 
> I show a Makefile hack below to disable it.
> 
> 
> 
> Fourth,
> On GNU/Linux coreutils automatically detects many supporting libraries
> (eg. xattr, selinux, libcap, etc.). If these libraries are only
> available as shared-libraries (and not static libraries), building will 
> fail.
> You can explicitly disable these libraries (shown below),
> or rebuild them with static components (which is beyond the scope of 
> this email).
> 
> ----
> 
> Having said all that, here's what worked for me:
> 
> 1.
> On Debian Stretch 9.8, gcc 6.3.0-18+deb9u1, coreutils 8.31,
> the following works:
> 
>      ./configure LDFLAGS="-static" --disable-xattr --disable-libcap \
>                  --disable-libsmack --without-selinux --without-gmp
>      make
> 
> Note that you'll see warnings such as these (as explained in item 2 above):
> ----
>    CCLD     src/groups
> src/src_ginstall-install.o: In function `get_ids':
> /home/gordon/projects/coreutils/src/install.c:602: warning: Using 
> 'getgrnam' in statically linked applications requires at runtime the 
> shared libraries from the glibc
>   version used for linking
> src/src_ginstall-install.o: In function `have_same_content':
> /home/gordon/projects/coreutils/src/install.c:154: warning: Using 
> 'endgrent' in statically linked applications requires at runtime the 
> shared libraries from the glibc
>   version used for linking
> src/src_ginstall-install.o: In function `need_copy':
> /home/gordon/projects/coreutils/src/install.c:220: warning: Using 
> 'getpwnam' in statically linked applications requires at runtime the 
> shared libraries from the glibc
>   version used for linking
> ----
> 
> But the binaries will be built successfully (and will be static).
> If you run them on the same system, they will automatically find
> the dynamic glibc, load it, and will be able to use the getpwnam(3)
> family of functions.
> 
> If you copy them to a different Linux-based system (not necessarily 
> glibc system), they will still work, except any user/group name-related
> functionality, e.g. on an Alpine-Linux system with user miles having 
> UID=1000:
> 
>    $ ./groups
>    ./groups: cannot find name for group ID 1000
>    1000
> 
>    $ ./chown miles 1
>    ./chown: invalid user: 'miles'
> 
>    $ ./chown 1000 1
>    [[ works ]]
> 
> 
> 
> 2.
> On same Debian 9.8 machine,
> the "musl" libc can be used to build fully static binaries ( 
> http://www.musl-libc.org/ ).
> A typical installation of "musl" on a GNU/Linux system provides
> a compiler-wrapper script called "musl-gcc" which can be used
> as $CC .
> 
> 
> A typical scenario will be (YMMV):
> 
>      wget http://www.musl-libc.org/releases/musl-1.1.21.tar.gz
>      tar -xzf musl-1.1.21.tar.gz
>      cd musl*
>      ./configure
>      make
>      make install   # into /usr/local/musl
> 
>      export PATH=/usr/local/musl/bin:$PATH
>      musl-gcc -v  # sanity check
> 
>      wget https://ftp.gnu.org/gnu/coreutils/coreutils-8.31.tar.xz
>      tar -xf coreutils-8.31.tar.xz
>      cd coreutils-8.31/
>      ./configure CC=musl-gcc LDFLAGS="-static"
>      make
> 
> This *almost* works, but will fail to link "libstdbuf" (as explained
> in item 3 above).
> It is easy to to modify the Makefile to simply disable building that
> library:
> 
>      sed -i '/pkglibexec_PROGRAMS/s|src/libstdbuf.so||' Makefile
>      make
> 
> And all the binaries will be built successfully, as static binaries.
> 
> 
> 
> 3.
> On Alpine Linux 3.9.2 (which uses "musl" libc natively, not glibc),
> the following "just works":
> 
>       ./configure CC=clang LDFLAGS="-static"
>       make
> 
> 
> 
> 4.
> In coreutils version 8.23 (released 2014) a new feature was added
> to create a single binary executable (conceptually similar to busybox).
> This could be useful for your situation.
> To use it, add "--enable-single-binary=symlinks" or 
> "--enable-single-binary=shebangs" to the "./configure" invocations above.
> 
> The result will be a "./src/coreutils" static binary,
> which will act as different coreutils programs based on invocation.
> 
> see 
> https://git.savannah.gnu.org/cgit/coreutils.git/diff/NEWS?id=71e2ea773414b2316bbe6b803b9a52c38d3752e8
>  
> .
> 
> 
> ---
> 
> Hope this helps,
> regards,
>   - assaf
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>