Re: Guile 2.2 .go files are larger

[Ludovic Courtès]

Hi!

Andy Wingo <address@hidden> skribis:

> On Sat 22 Apr 2017 15:19, address@hidden (Ludovic Courtès) writes:
>
>> The closure of Guix built with 2.0 is 193.8 MiB; when built with 2.2,
>> it’s 311.8 MiB.  Guix itself goes from 66 to 150 MiB:
>>
>> $ du -ms 
>> /gnu/store/jh07pwbyf5dbpdd5q0nvgagqkgmh76nh-guix-0.12.0-9.25a4/lib/guile/2.2
>> 101  
>> /gnu/store/jh07pwbyf5dbpdd5q0nvgagqkgmh76nh-guix-0.12.0-9.25a4/lib/guile/2.2
>> $ du -ms 
>> /gnu/store/rnpz1svz4aw75kibb5qb02hhccy2m4y0-guix-0.12.0-7.aabe/lib/guile/2.0
>> 24   
>> /gnu/store/rnpz1svz4aw75kibb5qb02hhccy2m4y0-guix-0.12.0-7.aabe/lib/guile/2.0
>
> Before we begin, some general notes.  My understanding is that the heap
> usage corresponding to an individual Guix process will be lower, both
> due to allocation of read-only data in read-only, shareable sections of
> the .go ELF files, allocation of read-write data in packed sections
> known to the GC but not managed by GC, and various optimizations that
> can eliminate or simplify some heap allocations (closure optimization
> among them).  In short my understanding is that Guile 2.2 (and systems
> built on it) should have a smaller run-time footprint than Guile 2.2.

Yes of course.

>> Would you have any suggestions to shrink the ELF files a bit?
>
> Why?  Have you compared gzipped or lzipped sizes?  I don't want to put
> effort in here that's not worth it :)

--8<---------------cut here---------------start------------->8---
$ du -ms $(./pre-inst-env guix pack address@hidden)
35      /gnu/store/25ppdmridc8i1j771s9c498y1sr9xfzb-tarball-pack.tar.gz
$ du -ms $(./pre-inst-env guix pack address@hidden)
40      /gnu/store/3fkhdpfpjjn8088fs0dxgb6hi38ac46m-tarball-pack.tar.gz
--8<---------------cut here---------------end--------------->8---

(Of course the difference is also due to the additional source code in
2.2.)

> Part of it is that our page alignment is 64 kB and so average wasteage
> per .go is 32kB, and there are a lot of .go files.  These are just zero
> bytes though.

OK.  It might be that file systems can store sparse files reasonably
efficiently.

> If you look at an individual file, you can use readelf to see things, or
> that old ELF visualizer I wrote:
>
>   https://wingolog.org/elf-mapper/
>
> Here's a map of gnu/packages/curl.go:
>
>   https://wingolog.org/pub/elf/elf-JWXYrI.png
>
> I think stripping the .debug_* stuff won't get you much of anywhere.
>
> Stripping arities info could reduce size by 10%.  We could figure out a
> different way to represent arities that takes less space.
>
> Compiling all the .go files together (requires Guile hacking) could
> remove that per-go 64kB alignment overhead.  Alternately we could do
> what GNU tools / ld.so do which maps the linear sequence of bytes in the
> file to aligned segments in memory.
>
> Honestly though I would punt.  It's not a run-time issue.  I don't think
> it's a transport issue.  It's only a disk space issue.  I personally
> don't plan to spend time on it but am happy to point out possibilities
> to people that will do so.

Sure, I understand.  I was just wondering if there was some low-hanging
fruit here.

Thanks for your feedback!

Ludo’.