bug#36511: extraneous recompiles of scm files while editing gnu/packages/

[Ludovic Courtès]

Robert Vollmert <address@hidden> skribis:

>> On 8. Jul 2019, at 12:03, Ludovic Courtès <address@hidden> wrote:

[...]

>> I was trying to address the “I see recompilation messages” issue that
>> you raised in the current framework.  I didn’t initially interpret the
>> bug report as a call for a new workflow.
>
> It’s not meant to be a call for a new workflow, either. The original
> suggestion from the “beyond 1.0” thread was a request for a focus on
> the developer experience by working on the quality of error messages and
> command output. One of the instances where that applies is with respect
> to guile recompile scam, as I remarked there.

OK, got it.  I initially viewed it as an immediate bug because I don’t
see those messages, but that’s because I’m so used to typing ‘make’ etc.

> Here’s another example, while working on a local checkout of a channel.
> Calling `guild compile` just makes things worse.
>
> $ rm -rf ~/.cache/guile
> $ guix build -L . -L ../guix-postgrest puzzledb-frontend
> ;;; note: source file ../guix-postgrest/bytestring.scm
> ;;;       newer than compiled 
> /gnu/store/sk1j6fkh855cm6ypp6799pgf8wvnlk76-postgrest/lib/guile/2.2/site-ccache/bytestring.go
> ;;; note: source file ../guix-postgrest/check.scm
> ;;;       newer than compiled 
> /gnu/store/sk1j6fkh855cm6ypp6799pgf8wvnlk76-postgrest/lib/guile/2.2/site-ccache/check.go

That’s bad (it’s even worse than having to type ‘make’ in the
development environment IMO because it’s a “user-facing” interface.)

How about the patch below?  That turns on auto-compilation, which is
probably a good thing in this context; as a side-effect, messages like
those above should disappear.

The only downside is ABI breakage: if we change the ABI of the <package>
record type (which is quite rare), then your channel code will no longer
run; you’ll get a message like:

  "<package>: record ABI mismatch; recompilation needed"

and you’ll have to “rm -rf ~/.cache/guile”.

Actually we could probably catch ‘record-abi-mismatch-error’ and
auto-compile when that happens.

Thoughts?

> $ rob@garp ~/guix-elm$ guild compile -L . -L ../guix-postgrest *.scm
> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;       or pass the --no-auto-compile argument to disable.
> ;;; compiling /run/current-system/profile/bin/guild
> ;;; compiled 
> /home/rob/.cache/guile/ccache/2.2-LE-8-3.A/gnu/store/9alic3caqhay3h8mx4iihpmyj6ymqpcx-guile-2.2.4/bin/guild.go
> wrote 
> `/home/rob/.cache/guile/ccache/2.2-LE-8-3.A/home/rob/guix-elm/versions.scm.go'
> wrote 
> `/home/rob/.cache/guile/ccache/2.2-LE-8-3.A/home/rob/guix-elm/elm.scm.go'
> wrote 
> `/home/rob/.cache/guile/ccache/2.2-LE-8-3.A/home/rob/guix-elm/elm2nix.scm.go’
> $ rob@garp ~/guix-elm$ guix build -L . -L ../guix-postgrest puzzledb-frontend

[...]

> ;;; note: source file ./elm.scm
> ;;;       newer than compiled 
> /gnu/store/amjb2461gsrcqiw4faifrf3vpyw46r36-elm/lib/guile/2.2/site-ccache/elm.go
> ;;; found fresh local cache at 
> /home/rob/.cache/guile/ccache/2.2-LE-8-3.A/home/rob/guix-elm/elm.scm.go

Bah, Guile is really talkative.

We could redirect Guile’s ‘current-warning-port’ to the bit-bucket
wholesale, but we’d also be missing out on more useful info such as
compiler warnings.

Unfortunately, even if we monkey-patch specifically
‘primitive-load-path’, we’re silencing too much:

  (set! primitive-load-path
    (let ((do-load-path primitive-load-path))
      (lambda* (file #:optional (exception? #t))
        (parameterize ((current-warning-port (%make-void-port "w")))
          (do-load-path file exception?)))))

Not sure what can be done on the Guix side.

Thoughts?

Thanks,
Ludo’.

diff --git a/guix/discovery.scm b/guix/discovery.scm
index 5bb494941b..76b58b7688 100644
--- a/guix/discovery.scm
+++ b/guix/discovery.scm
@@ -93,6 +93,16 @@ DIRECTORY is not accessible."
                               directory (strerror errno)))
                     '())))))
 
+(define-syntax-rule (with-auto-compilation exp ...)
+  (let ((compile? %load-should-auto-compile))
+    (dynamic-wind
+      (lambda ()
+        (set! %load-should-auto-compile #t))
+      (lambda ()
+        exp ...)
+      (lambda ()
+        (set! %load-should-auto-compile compile?)))))
+
 (define* (scheme-modules directory #:optional sub-directory
                          #:key (warn (const #f)))
   "Return the list of Scheme modules available under DIRECTORY.
@@ -103,19 +113,22 @@ name and the exception key and arguments."
   (define prefix-len
     (string-length directory))
 
-  (filter-map (lambda (file)
-                (let* ((file   (substring file prefix-len))
-                       (module (file-name->module-name file)))
-                  (catch #t
-                    (lambda ()
-                      (resolve-interface module))
-                    (lambda args
-                      ;; Report the error, but keep going.
-                      (warn module args)
-                      #f))))
-              (scheme-files (if sub-directory
-                                (string-append directory "/" sub-directory)
-                                directory))))
+  ;; Turn on auto-compilation so that user modules hit via
+  ;; %PACKAGE-MODULE-PATH are automatically compiled.
+  (with-auto-compilation
+   (filter-map (lambda (file)
+                 (let* ((file   (substring file prefix-len))
+                        (module (file-name->module-name file)))
+                   (catch #t
+                     (lambda ()
+                       (resolve-interface module))
+                     (lambda args
+                       ;; Report the error, but keep going.
+                       (warn module args)
+                       #f))))
+               (scheme-files (if sub-directory
+                                 (string-append directory "/" sub-directory)
+                                 directory)))))
 
 (define* (scheme-modules* directory #:optional sub-directory)
   "Return the list of module names found under SUB-DIRECTORY in DIRECTORY.
diff --git a/guix/ui.scm b/guix/ui.scm
index 7d6ab9a2a7..f87b29d0fc 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -1668,6 +1668,9 @@ and signal handling has already been set up."
   ;; number of 'stat' calls per entry in %LOAD-PATH.  Shamelessly remove it.
   (set! %load-extensions '(".scm"))
 
+  ;; The two-line auto-compilation message doesn't bring much so just hide it.
+  (set! %warn-auto-compilation-enabled (const #t))
+
   (match args
     (()
      (format (current-error-port)