Re: Parallel (de-)compression

[Ludovic Courtès]

Andreas Enge <address@hidden> skribis:

> On Fri, Dec 04, 2015 at 03:44:38PM +0100, Ludovic Courtès wrote:
>> Are you saying that xz-compressing TeX Live to resend it to
>> hydra.gnu.org is too CPU-intensive?
>
> That depends on your definition of "too". In any case, on the Novena board
> with an SSD attached, CPU is the limiting factor during this phase,
> pushing the CPU load on one core to 100% (while the other cores are idle).

OK.

>> Another option would be to trade compression ratio for reduced CPU usage
>> by using, say, ‘xz -2’ or ‘gzip’.
>> We did something similar in 5ef9d7d to reduce CPU consumption on the
>> front-end.  Usually it’s much less important to reduce CPU consumption
>> on the build machines, but your experience seems to suggest otherwise.
>
> If possible, it would be more interesting to leverage the several cores
> and not make sacrifices on the compression quality.

It’s not necessarily the best option to increase throughput: the build
machine may be busy building other things, and thus unable to dedicate
all its cores to compression.

Dunno.

Ludo’.