[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Parallel downloads
From: |
Ludovic Courtès |
Subject: |
Re: Parallel downloads |
Date: |
Sun, 17 Nov 2019 18:52:13 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) |
Hi Mark,
Mark H Weaver <address@hidden> skribis:
> For these reasons, I'm inclined to think that parallel downloads is the
> wrong approach. If a single download process is not making efficient
> use of the available bandwidth, I'd be more inclined to look carefully
> at why it's failing to do so. For example, I'm not sure if this is the
> case (and don't have time to look right now), but if the current code
> waits until a NAR has finished downloading before asking for the next
> one, that's an issue that could be fixed by use of HTTP pipelining,
> without multiplying the memory usage.
I agree. There’s HTTP pipelining for narinfos but not for nars. Worse,
before fetching a nar, we do a GET /nix-cache-info, and in fact we spawn
a new ‘guix substitute’ process for each download (for “historical
reasons”).
So there’s room for optimization there!
Ludo’.
- Re: Parallel downloads, (continued)
- Re: Parallel downloads, Leo Famulari, 2019/11/12
- Re: Parallel downloads, Ludovic Courtès, 2019/11/17
- Re: Parallel downloads, Mark H Weaver, 2019/11/13
- Re: Parallel downloads, Pierre Neidhardt, 2019/11/13
- Re: Parallel downloads, Leo Famulari, 2019/11/13
- Re: Parallel downloads, Pierre Neidhardt, 2019/11/13
- Re: Parallel downloads,
Ludovic Courtès <=
- Re: Parallel downloads, Bengt Richter, 2019/11/06