Re: Conda environments and reproducibility

[Ricardo Wurmus]

"Lestang, Thibault" <t.lestang@imperial.ac.uk> writes:

> Ludovic Courtès <ludovic.courtes@inria.fr> writes:
>
>> Any findings so far?  Looking at the pipelines, it seems to be all
>> green, right?
>
> Timely reply - all green until 3 days ago when the job timed out after 70min. 
> However, I re-ran the job manually this morning and it succeeded within a 
> couple of minutes. Not 
> quite sure what happened but probably not related to conda. Not logs 
> available unfortunately.
>
> If the process of reproducing the environment is going to fail at some point, 
> I 
> wonder if we could accelerate this process by defining a more complex 
> environment. 
> Any ideas?

A more complex environment would increase the chance of failure because
it increases the complexity of the challenge to the resolver.  While it
would be a useful demonstration to see the resolver fail I think it is
the least damning kind of failure.

As Simon suggests, changing the underlying system that *currently*
satisfies all the implicit assumptions that Conda artefacts contain
would likely yield a more realistic and interesting kind of failure.

>
> Simon Tournier <zimon.toutoune@gmail.com> writes:
>
>> 1. also use the image continuumio/miniconda3:latest
>> 2. install Miniconda on the top of the Docker image of Debian
>>   unstable and run "apt update && apt upgrade"
>> 
>> And I expect that #2 will break first, then #1 and last the current
>> one.
>
> Could you elaborate on this? For context the current pipeline 
> pulls a pinned miniconda image then updates conda (=conda update conda=).  
> Do you expect system libraries (I mean software installed through apt, not 
> managed by conda) to influence the conda environment creation?  My current 
> understanding is that conda brings its  own copies of these libraries without 
> relying 
> on whatever was/will be installed through other ways (e.g. apt).

This depends on the packages.  There are packages that do link with
system libraries, and these are provided by a base image in which the
binary artefacts are built.

-- 
Ricardo