[gwl-devel] Next steps for the GWL

[Ricardo Wurmus]

Hi,

I’m going to use the GWL in the next few days to rewrite the PiGx RNAseq
pipeline from Snakemake.  This will likely show me what features are
still missing from the GWL and what implemented features are awkward to
use.

My goals for the future (in no particular order) are as follows:

* add support for running workflows from a file (without
  GUIX_WORKFLOW_PATH)

* replace the “qsub”-based Grid Engine execution engine with proper
  DRMAA bindings

* add support for running remote jobs via Guile SSH

* add support for spawning jobs on AWS or Azure or via Kubernetes

* tighter integration with Guix features, e.g. to export a container
  image per process via “guix pack” or to pack up the whole workflow as
  a relocatable executable.

* inversion of control: enable workflow designers to use the GWL as a
  library, so that the “guix workflow” user interface does not need to
  be used at all (see PiGx for an example).

* explore the use of inferiors — the GWL should be usable with any
  version of Guix that may be installed, not just the version that was
  used at compilation time.  Can we use “guix repl” and inferiors,
  perhaps?

* add support for executing processes in isolated environments
  (containers) — this requires a better understanding of process inputs.

What do you think about this?  Who would like to help flesh out these
ideas?

--
Ricardo