[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [gwl-devel] support for containers
From: |
zimoun |
Subject: |
Re: [gwl-devel] support for containers |
Date: |
Wed, 30 Jan 2019 11:17:02 +0100 |
Hi Ricardo,
On Wed, 30 Jan 2019 at 00:16, Ricardo Wurmus <address@hidden> wrote:
> Since we don’t hash the data (because it’s expensive) the scripts are
> “proxies” for the data files. We compute the hashes over the dependent
> scripts and assume that this is enough to decide whether to recompute
> data files or to serve them from the cache/store.
Just to be sure to well understand your point, let pick the simple
example from genomics pipeline:
FASTQ -align-> BAM -variant-> VCF
So, you intend to hash:
- the data FASTQ
- the scripts align and variant
Or only the scripts containing reference to inputs (here FASTQ), where
the reference is a location fixed by the user.
Well, hashing the scripts and assuming they "mirror" the data files
appear to me an efficient design for the CAS.
--
simon
- [gwl-devel] support for containers, Ricardo Wurmus, 2019/01/28
- Re: [gwl-devel] support for containers, Ricardo Wurmus, 2019/01/29
- Re: [gwl-devel] support for containers, zimoun, 2019/01/29
- Re: [gwl-devel] support for containers, Ricardo Wurmus, 2019/01/29
- Re: [gwl-devel] support for containers, zimoun, 2019/01/29
- Re: [gwl-devel] support for containers, Ricardo Wurmus, 2019/01/30
- Re: [gwl-devel] support for containers, zimoun, 2019/01/29
- Re: [gwl-devel] support for containers, Ricardo Wurmus, 2019/01/29
- Re: [gwl-devel] support for containers,
zimoun <=
- Re: [gwl-devel] support for containers, Ricardo Wurmus, 2019/01/30
Re: [gwl-devel] support for containers, zimoun, 2019/01/29