My Guile TODO list

[Mark H Weaver]

Hello all,

I occasionally talk about my large Guile TODO list, and sometimes people
say that I should put it somewhere public.  Okay, here it is (not
including my ticked messages in Gnus).  It includes some items that are
probably controversial, especially near the end, and I'd rather not get
stuck in huge threads about those ideas right now, so please just take
this for what it is: a place where I record tentative ideas so they
won't be forgotten.  Honestly, I'll be lucky if I find time to do a
small fraction of this stuff :)

    Best,
     Mark


Guile TODO
==========

* Refactor pending numerics patches (Improved rationals et al)
  (This will help Neils Möller with his minigmp project.)

* Get rid of all of those 'resolve-module' and 'resolve-interface' calls
  in hot procedures of (language elisp runtime)! (master)

* Optimize important elisp primitives (master)

* VM instruction to lisp canonicalize (#f or '() => #nil) ?

* How to modularize libunicode?

* Fix the warning system

* Add call/ec to stable-2.0

* Make call/ec as fast as possible in master (VM instruction?)

* Allow the ,d REPL command to work for syntactic keywords

* Improve (ice-9 match) documentation.

* [SUBMITTED] Faster gensym: postpone giving it a name until requested.

* Optimize exact-integer-sqrt: use floating-point for integers less than
  2^53 (I learned that this is reliable from Shiro Kawai of Gauche)
  (Related idea: use floating-point for some larger subset of the
  rationals.  When is it safe to trust the result?)

* Improve the implementations of map, for-each, et al.

* How to modularize the numerics code?
* How to implement the numerics operation dispatchers?
  Best answer: Use GOOPS + FFI for GMP/MPFR/MPC; if not fast, fix it.
  Practical answer: ?

* Look into this:
   UNRESOLVED: r6rs-arithmetic-fixnums.test: fx+/carry: simple
   UNRESOLVED: r6rs-arithmetic-fixnums.test: fx-/carry: simple
   UNRESOLVED: r6rs-arithmetic-fixnums.test: fx*/carry: simple

* R7RS compliance
   * optional ellipsis specifier for syntax-rules et al
   * 'syntax-error'
   * define-values
   * let-values and let*-values (without loading SRFI-11)
   * include-ci
   * bytevectors in core R7RS
   * lazy and eager (without loading SRFI-45)
   * |...| symbol notation, and \xXX within symbols
   * support #\escape and "\escape"
   * support \xXXXX in string literals
   * allow whitespace between \ and newline in string literals
   * #!fold-case and #!no-fold-case
   * #true and #false
   * datum labels for circular and shared substructures
   * nan? and finite? now accept complex numbers
     (should probably change inf? and infinite? as well)
   * exact-integer?
   * R7RS exceptions
   * make sure define-record-type is R7RS compliant
   * optional third parameter to 'member' and 'assoc'
   * define-library
   * digit-value
   * char-foldcase
   * string-ni=? et al
   * vector->string and string->vector
   * vector-copy supports optional (start end fill) args
   * vector-fill! supports optional (start end) args
   * bytevector-copy! with 3 args
   * bytevector-copy-partial{,!}
   * write bytevectors with #u8 (and elements in hex) by default?
   * {map,for-each} stops when shortest list runs out
   * string-{map,for-each} accepts multiple strings
   * vector-{map,for-each}
   * make sure {map,vector-map,string-map} are multi-return safe
   * {scheme-report,null}-environment for R7RS
   * 'environment'
   * 'port-open?'
   * R7RS binary ports and bytevector ports
      * {textual,binary}-port?
      * open-binary-{input,output}-file
      * open-{input,output}-bytevector
      * get-output-bytevector
      * {read,peek,write}-u8, u8-ready?
      * read-bytevector{,!}
      * write-bytevector
      * write-partial-bytevector
   * read-line
   * write-simple
   * flush-output-port
   * load with optional 'environment specifier' as second argument
   * get-environment-variable{,s}
   * current-{second,jiffy}, jiffies-per-second
   * R7RS feature identifiers: r7rs, exact-closed, ratios, exact-complex,
     ieee-float, full-unicode, windows, posix, unix, darwin, linux, bsd,
     freebsd, solaris, i386, x86-64, ppc, sparc, jvm, clr, llvm, ilp32,
     lp64, ilp64, big-endian, little-endian, guile, guile-2, guile-2.0

* catch and raise in terms of R7RS/R6RS exceptions/conditions

* compile-time dependency tracking for .go files

* Good implementation of empty case-lambdas

* Track down bytevector test failures reported by Hans Aberg

* Add test cases:
    string mutation test case (segfaults on earlier versions)
    attempt to mutate empty range of immutable string
    narrow substring of wide string should be narrow
    empty substring should not hold reference to original stringbuf
    newly allocated empty strings share a common null stringbuf
    make sure multiple copies of the empty string are distinct
      from C
      from Scheme

* Improve the hash function, and allow GOOPS objects et al to
  declare their own custom hash function.

* Improve the PRNG and its initializer

* Improve ice-9/format
   * truncate/ <= quotient and remainder [DONE but not committed]
   * Exact numbers in positional-number system without imprecision

* Add missing documentation for several interfaces in foreign.c

* Consistently validate that indices are _exact_ integers.
  (XXX maybe already done?)

* Consistently enforce the immutability or literals (lists, vectors,
  strings, syntax, and ideally in primitive-eval as well)

* Top-level identifiers introduced by macros should be hygienic

* (?) investigate the feasibility of making 'equal?' and 'eqv?' as fast
  as 'eq?' when at least one of two arguments is immediate-or-interned,
  by making it easy to determine from the bit pattern of the SCM value
  itself whether the value is immediate-or-interned.  This would require
  changing the representation of symbols and keywords to immediate
  values (containing an index into a table).

* Numerics improvements:
   * Add scheme-accessible frexp and ldexp procedures
   * Add exact-integer? predicate
   * exact-integer-root with remainder
   * Exact 'expt' to rational power when the result is rational
   * Exact log-floor, log10(?), and log2(?) (or arbitrary base)
   * Deprecate low-level numeric predicates in 2.2
   * Improved modularity
   * Extensible numerics with GOOPS
   * Arbitrary-precision floats in Scheme with GOOPS
   * Complex numbers with general real components
   * Infinite-precision reals (maybe continued fractions/logs)
   * Scmutils for Guile 2
   * Modular MPFR/MPC support

* Filesystem paths as sequences of path components (et al)

* Strings overhaul

  * Decide on strategy for POSIX byte strings
     * Decide on implementation
        * Distinct low-level type
          (implied by strings-as-views-of-bytevectors/bytestrings)
        * UTF-8 with special range of codepoints
     * Decide on semantics
        * (bytestring? x) implies (string? x)
        * (bytestring? x) or (string? x) implies (generalized-string? x)

   * Strings as views of bytevectors (UTF-8 or POSIX byte strings)
      Note that bytevectors can use any range of aligned(?) memory,
        with no inline header or footer, in the middle of some
        arbitrary 'parent' block.  (Thanks, Andy! :)

   * UTF-8 backed strings, while allowing (external?) code to easily
     work in terms of bytes instead of characters

   * (?) Immutable strings built by snapshotting a mutable string,
     (to allow the user to dynamically create an immutable string
     that supports efficient append et al)

   * (?) Mutable strings as mutable pointers to immutable strings

   * (?) Efficient data structure for large strings (and similarly for
         bytevectors/bytestrings)

* Allow 'with-{input-from,output-to}-file' et al to accept file open
  modes, e.g. encoding=?, binary, create, append, truncate, etc.

* (ice-9 popen) overhaul
   * Support closing output side of pipe before closing input side
   * (?) Maybe honor current-{input,output,error}-port settings
   * (?) Integrate more features (maybe from Scsh)

* Ports overhaul
   * (?) Remove scm_t_port from the ABI
   * Character ports as views of byte ports
     (maybe allow interleaved byte/char access)
   * Bytevector ports
   * Efficient soft ports

* Chunked encoding for web client

* Sweet expressions

* Gnulib-style layer shipped with applications to allow use of Guile 2
  APIs with transparent 1.8 compatibility
   * (Ideally) get modules accepted into Gnulib itself

* New bracket pairs (maybe allow user to define them),
  maybe reduced to macro calls,
  e.g. {a b c} => (within-curly-brackets a b c)

* Parser combinators a.l.a. Haskell

* Efficient purely-functional data structures,
  like Chris Okasaki's Edison, for Guile

* Monad library?

* Improve error messages and debugging
   * Improve error messages, searching for bad messages
   * Learn from MIT/GNU Scheme
   * Improve backtraces and debugging
      * History rings for tail calls
      * Display in 'current subproblem' form
      * (?) Continuation as a psuedo-expression with a hole

* Improve evaluator
   * (?) Just use compiler with most optimizations disabled?
   * Improve error messages
   * Improve debugging

* Plotting software using guile-cairo

Big picture items
=================

* (?) unexec

* Register-based VM

* Improved modularity

* Translate as much C code as possible to Scheme.

* Move as much as possible into a shared read-only segment of libguile
   * Purge as much *_init activity as possible
   * Transition to auto-generated C files with statically initialized data
     structures (i.e. enhance Snarfing)

* Numbers overhaul
   * Improved modularity; allow a minimal core
   * Make GOOPS fast enough that it can replace hand-optimized C
     dispatchers in numbers.c
   * Support and implement many advanced arithmetic objects
      * Abstract algebra: Rings, Commutative Rings, Fields, etc.
      * Arbitrary-precision floating-point (with unbounded exponent)
      * MPFR
      * Infinite-precision reals (e.g. continued fractions/logs)
      * Exact arithmetic numbers
      * Exact irrationals (some useful subset)
         (?) maybe just use symbolic expressions here
      * Real Intervals
      * Complex numbers with arbitrary rectangular coordinates
        (e.g. exact rationals)
      * MPC
      * Quaternions
      * Vectors
      * Matrices
      * Polynomials
      * Symbolic expressions based on lexical scoping and
        syntax-objects

* Switch most Scheme primitives to use our own calling convention, and
  provide (optional) auto-generated wrappers that use the standard C
  calling convention.

* Compiler overhaul
   * Immutable (or rarely mutated) top-level bindings
   * Immutable pairs?
   * Tracing and hotspots drive the incremental optimizer

* FFI overhaul
   * Native code generation where possible, else generated C code
   * Fast enough so that it could be our only link to C, and can be used
     to wrap all scm_*, allowing all of the C sources to be more nicely
     written, and supporting automatic type checking.