[Chicken-users] A Scheme based make - soliciting you comments

[Jörg F . Wittenberger]

Hi all,

these days I ran (again as every once in a while) a case which made me 
longing for a make(1) in Scheme. Gave the make egg a try and… decided I'd 
need something else. Something powerful enough to make it easier to 
maintain Chickens build and similar complex things.

So far I ended up with something working.  I'm not yet sure that it's worth
the effort to document/release it, so please tell me if you like it.
Furthermore, and even more important: there's a bit of syntax defined.
In attempt to avoid the worst pitfalls
(as in The Three Laws of Programming Language Design
http://lambda-the-ultimate.org/node/4754 )
I'd love to read your bitching on the following.

Best Regards

/Jörg

The rest is the "scheme makefile" for the "scheme make" itself. Insofar 
it's working. (Just the "windows" part is fake; only used unix so far. The 
conditional stuff however works.)

;; The following line retrieves the tharget to make from the environment
;; variable "make" using "ssx" as default value.
#(make: "make" "ssx")

;; The syntax of the following is really subject to discussion.
;;
;; It is essentially equivalent to
;;
;; (define FEAT_ORIG "v1"
;;
;; A different syntax seems to be in order because in constrast to the
;; semantics of a normal `define`, the *first* definition seen is
;; used; no overwrites.  Additionally there is an environment lookup
;; to "FEAT_ORIG"; if the environment variable is set, it determines
;; the value.  The value "v1" just a default.
#(param: FEAT_ORIG "v1") ;; expose -feature arg to call site

#(param: PLATFORM "unix" "or windows, see below")

;; Expose tracing to potential overwrite by invocation.
#(param: CSCTRACE '("-no-trace" "-no-lambda-info"))

;; `define-dir` and `define-file` declare binding to some directory or
;; file.  This is mostly useful to avoid the system's directory
;; separator showing up in the source code.
(define-dir PMPTH "..")                       ; path to pre-made stuff
(define CSCINCLUDE `("-I" ,PMPTH))

(define CSC "csc")

(define cscflags.opt `("-O3" ,CSCTRACE))

;; conditional ex-/inclusion
;;
;; Comments on the syntax to be used again very much solicited.
;;
;; * Conditionals are ONLY available at top-level and must find their
;;   corresponding #(end) token in the same file.  For rationale see
;;   SRFI-0.
;; * This syntax is NOT in s-expressions.  Rationale: Be friendly
;;   diff(1) and maintainers.  Those are often late additions or even
;;   temporary measurements.  Good practise is to adhere indentiation
;;   rules (often done by automatic means).  If this would introduce
;;   additional nesting, too much un-changed source ends up in the
;;   diff.
;; * By now there is no "else" clause.  Convince me that it's good to
;;   have.  So far I feel it's clearer to enforce all cases to be
;;   stated explicit.  Users who really need an "else" can always do
;;   via variables beeing defined in the individual branches.

#(if: (equal? PLATFORM "unix") )
(define ext.obj "o")
#(end)
#(if: (equal? PLATFORM "windows") )
(define ext.obj "obj")
#(end)


(define (csc x . o)
 ;; `result-file` and `source-file` are not mandatory to be used.
 ;; Those are convinience-and-saftey related: they return their
 ;; argument; as side effect the file is registered as source or
 ;; result.  An exception is raised upo attempt to overwrite a file
 ;; declared with the source property.
 (let ((o (result-file
            (if (pair? o) (car o) (filename #f x ext.obj)))))
   (run CSC cscflags.opt CSCINCLUDE '-J "-c" x
         '-emit-type-file (result-file (filename #f x "types"))
         "-o" o)))

;; Same as above, but with -compile-syntax .
(define (csc/syntax module into)
 (run `(,CSC -feature ,FEAT_ORIG -O3 -c ,module
              -compile-syntax
              ,CSCINCLUDE -J
              -emit-type-file ,(result-file (filename #f module "types"))
              -o ,(result-file into))))

;; Comments may use Scheme syntax, even though this loks weird here.
#; #(Imported objects not (yet) build here.)
(define SSX_PM_OBJECTS
 (map
  ;; `filename` is some crazy syntax using decompose-pathname,
  ;; make-pathname and friends behind the scene.  It accepts symbols
  ;; instead of strings for components and flattens nested lists in
  ;; the path for convenience.  (You don't want them here and doing
  ;; the right thing in plain Scheme is just an error prone, tedious
  ;; task whose only outcome is source clutter in this context.)
  (lambda (o) (filename PMPTH o "o"))
  '(srfi-34 srfi-35 matchable)))

(define SSX_OBJECTS
 (map
  (lambda (o) (filename #f o 'o))
  '(alexpander "synclo" ssx-divert make)))

;; The `make(1)` alike part.  Declares a target, it's dependencies and
;; a list of commands (here only one) to build it.  Nested lists are
;; in the depenciencies flattened secretly.
;;
;; `make-rule` is syntax.  It will only register the rule.  No
;; immediate actions (except for some validation).  Actions are run
;; once all input files are evaluated.  ((This leads to the question
;; whether or not we should allow redefinition of variables at all.
;; At worst it's confusing while there is little to gain as far as I
;; can see by now.))
;;
;; More Questions:
;; * Suggestions for be a better name?
;; * Like any normal syntax this is avail everywhere, but only useful
;;   at top-level.  Should we enforce it to appear only at top-level?
(make-rule "ssx"
 `("ssx.scm" ,SSX_PM_OBJECTS ,SSX_OBJECTS)
 (run CSC "-O0" CSCTRACE CSCINCLUDE "ssx.scm" SSX_PM_OBJECTS SSX_OBJECTS
      "-static-libs" "-o" "ssx"))

(define-dir syntax.dir '("syntax" hygnc))

;; `define-source` and `define-result` are syntactic sugar around
;; `define` together with `source-file` and `result-file` respectively
;; plus `filename` (see above).
(define-source alexpander.module
 syntax.dir 'mod-chkn-alexpander 'scm)
(define-source alexpander.code
 syntax.dir 'alexpander "scm")

(make-rule
"alexpander.o"        ;; yeah, "we can" just use the plain file name for
                ;; brevity; define-source etc. is somewhat optional.
(list alexpander.module alexpander.code)
(csc/syntax alexpander.module "alexpander.o"))

;; This example doen't make much sense here.  However if you look into
;; `make.rules` from the chicken build, it's easy to see how this
;; could simplify and shorten the source while adding saftey belts.
(define-syntax define-my-sources
 (syntax-rules ()
   ((_ name dir ext s ...)
    (define name
      (letrec
           ((dfsr (lambda (f) (if (pair? f) (map dfsr f) (dfs f))))
            (dfs (lambda (f) (define-source t dir f ext) t)))
         (map dfsr (list s ...)))))))

;; BTW: don't ask why "alexpander" and "syntactic closures" are here
;; at all.  It's heritage.  The whole idea grew within a tool quickly
;; hacked together to debug hygienic macros.  At this point I learned
;; that at least chicken's syntactic closures and alexpander will a)
;; each catch different errors b) happily compile some broken code and
;; c) produce wrong results for (probably) correct input.  Thus I ran
;; my macros through all of them and used the intersection of things
;; actually passing and working in all cases.
;;
;; I might have to take this part of the source out before finalizing
;; the tool.  At the other hand it might be really useful to many to
;; just leave it in.  Aside: how would reliably fully expand via
;; chicken and pretty-print the results?  Would be really helpful.

(define-my-sources
 synclo.mod.src
 syntax.dir "scm"
 'mod-chkn-synclo)

(define-my-sources
 synclo.src
 `(,syntax.dir "synclo") "scm"
 'syntactic-closures)

(define-result synclo-object-code "synclo" "o")

(make-rule
synclo-object-code `(,synclo.mod.src ,synclo.src)
;; If define-my-sources where smarter, we could avoid `car` here.
(csc (car synclo.mod.src) synclo-object-code)
)

(define-my-sources
 ssx-divert.src
 '("bldutls") "scm"
 "ssx-divert")

;; Dealing with auxillary result files.  ".import.scm" is for a good
;; reason only written if it actually changed.  Beware how to use it
;; to not enforce useless recompilation upon re-invocation after
;; successful compile operations.
(define-result ssx-divert.import #f 'ssx-divert "import.scm")

(make-rule
(list
 (filename #f "ssx-divert" ext.obj)
;; It that's here, ssx-divert is re-made even though the exports did
;; not change.
;;
;;  ssx-divert.import
 )
ssx-divert.src
(csc (car ssx-divert.src)))

;; As a single rule it will not trigger a rebuild.  (TODO figure out
;; if this special handling could be avoided without breaking other
;; things.)
(make-rule ssx-divert.import "ssx-divert.o")

(define-my-sources
 make.src
 '("bldutls") "scm"
 'make)

(make-rule
(filename "make" ext.obj)
`(,make.src ,ssx-divert.import)
(csc/syntax (car make.src) (filename #f "make" ext.obj)))

;; Now some sugar.  Maintaining a "manifest" file and the content of
;; the distro is often tedious.  All too often I just found out what's
;; missing by running a fresh build from the distributed files within
;; an empty directory.  This should help.

;; Additional source files (documentation etc.).
(define-source ssx-makefile #f 'mk 'scm)
(define dist-files
 ;; Remove souces not to include in the distribution.
 (filter
  (lambda (x) (not (memq x SSX_PM_OBJECTS)))
  ;; (all-source-files) returns all sources, be them explicit
  ;; declared or implied by their use as a dependency.
  (all-source-files)))

(make-rule "tar" dist-files (run "tar" "-czf" "ssx.tar.gz" dist-files))


...................