master 79e2d04: Document native-compilation

[Eli Zaretskii]

branch: master
commit 79e2d0486c89c6c452df33792976ea6fdf825185
Author: Eli Zaretskii <eliz@gnu.org>
Commit: Eli Zaretskii <eliz@gnu.org>

    Document native-compilation
    
    * doc/lispref/loading.texi (How Programs Do Loading)
    (Library Search): Update for native-compilation features.
    * doc/lispref/compile.texi (Native Compilation)
    (Native-Compilation Functions, Native-Compilation Variables): New
    chapter and sections.
    * doc/lispref/elisp.texi (Top): Update the top-level menus.
    
    * etc/NEWS: Add a reference to the ELisp manual.
---
 doc/lispref/compile.texi | 245 +++++++++++++++++++++++++++++++++++++++++++++++
 doc/lispref/elisp.texi   |   6 ++
 doc/lispref/loading.texi |  33 ++++++-
 etc/NEWS                 |   5 +-
 4 files changed, 286 insertions(+), 3 deletions(-)

diff --git a/doc/lispref/compile.texi b/doc/lispref/compile.texi
index bf42004..f078779 100644
--- a/doc/lispref/compile.texi
+++ b/doc/lispref/compile.texi
@@ -793,3 +793,248 @@ The @code{silly-loop} function is somewhat more complex:
 17  return                ; @r{Return value of the top of stack.}
 @end group
 @end example
+
+@node Native Compilation
+@chapter Compilation of Lisp to Native Code
+@cindex native compilation
+@cindex compilation to native code (Emacs Lisp)
+
+@cindex native code
+  In addition to the byte-compilation, described in @ref{Byte
+Compilation, previous chapter}, Emacs can also optionally compile Lisp
+function definitions into a true compiled code, known as @dfn{native
+code}.  This feature uses the @file{libgccjit} library, which is part
+of the GCC distribution, and requires that Emacs be built with support
+for using that library.  It also requires to have GCC and Binutils
+(the assembler and linker) available on your system for you to be able
+to native-compile Lisp code.
+
+@vindex native-compile@r{, a Lisp feature}
+  To determine whether the current Emacs process can produce and load
+natively-compiled Lisp code, test whether the @code{native-compile}
+feature is available (@pxref{Named Features}).  Alternatively, call
+@code{native-comp-available-p} (@pxref{Native-Compilation Functions}).
+
+  Unlike byte-compiled code, natively-compiled Lisp code is executed
+directly by the machine's hardware, and therefore runs at full speed
+that the host CPU can provide.  The resulting speedup generally
+depends on what the Lisp code does, but is usually 2.5 to 5 times
+faster than the corresponding byte-compiled code.
+
+  Since native code is generally incompatible between different
+systems, the natively-compiled code is @emph{not} transportable from
+one machine to another, it can only be used on the same machine where
+it was produced or on very similar ones (having the same CPU and
+run-time libraries).  The transportability of natively-compiled code
+is the same as that of shared libraries (@file{.so} or @file{.dll}
+files).
+
+  Libraries of natively-compiled code include crucial dependencies on
+Emacs Lisp primitives (@pxref{What Is a Function}) and their calling
+conventions, and thus Emacs usually won't load natively-compiled code
+produced by earlier or later Emacs versions; native compilation of the
+same Lisp code by a different Emacs version will usually produce a
+natively-compiled library under a unique file name that only that
+version of Emacs will be able to load.  However, the use of unique
+file names allows to have in the same directory several versions of
+the same Lisp library natively-compiled by several different versions
+of Emacs.
+
+@vindex no-native-compile
+  A non-@code{nil} file-local variable binding of
+@code{no-byte-compile} (@pxref{Byte Compilation}) also disables the
+native compilation of that file.  In addition, a similar variable
+@code{no-native-compile} disables just the native compilation of the
+file.  If both @code{no-byte-compile} and @code{no-native-compile} are
+specified, the former takes precedence.
+
+@menu
+* Native-Compilation Functions::  Functions to natively-compile Lisp.
+* Native-Compilation Variables::  Variables controlling native compilation.
+@end menu
+
+@node Native-Compilation Functions
+@section Native-Compilation Functions
+@cindex native-compilation functions
+
+   Native-Compilation is implemented as side effect of
+byte-compilation (@pxref{Byte Compilation}).  Thus, compiling Lisp
+code natively always produces its byte code as well, and therefore all
+the rules and caveats of preparing Lisp code for byte compilation
+(@pxref{Compilation Functions}) are valid for native-compilation as
+well.
+
+   You can natively-compile either a single function or macro
+definition, or a whole file of Lisp code, with the
+@code{native-compile} function.  Natively-compiling a file will
+produce both the corresponding @file{.elc} file with byte code and the
+@file{.eln} file with native code.
+
+@findex native-comp-limple-mode
+@vindex native-comp-verbose
+  Native compilation might produce warning or error messages; these
+are normally recorded in the buffer called
+@file{*Native-compile-Log*}.  In interactive sessions, it uses the
+special LIMPLE mode (@code{native-comp-limple-mode}), which sets up
+@code{font-lock} as appropriate for this log, and is otherwise the
+same as Fundamental mode.  Logging of messages resulting from
+native-compilation can be controlled by the @code{native-comp-verbose}
+variable (@pxref{Native-Compilation Variables}).
+
+  When Emacs is run non-interactively, messages produced by
+native-compilation are reported by calling @code{message}
+(@pxref{Displaying Messages}), and are usually displayed on the
+standard error stream of the terminal from which Emacs was invoked.
+
+@defun native-compile function-or-file &optional output
+This function compiles @var{function-or-file} into native code.  The
+argument @var{function-or-file} can be a function symbol, a Lisp form,
+or a name (a string) of the file which contains the Emacs Lisp source
+code to compile.  If the optional argument @var{output} is provided,
+it must be a string specifying the name of the file to write the
+compiled code.  Otherwise, if @var{function-or-file} is a function or
+a Lisp form, this function returns the compiled object, and if
+@var{function-or-file} is a file name, the function returns the full
+absolute name of the file it created for the compiled code.  The
+output file is by default given the @file{.eln} extension.
+
+This function runs the final phase of the native compilation, which
+invokes GCC via @file{libgccjit}, in a separate sub-process, which
+invokes the same Emacs executable as the process that called this
+function.
+@end defun
+
+@defun batch-native-compile
+This function runs native-compilation on files specified on the Emacs
+command line in batch mode.  It must be used only in a batch execution
+of Emacs, as it kills Emacs upon completion of the compilation.  If
+one or more of the files fail to compile, the Emacs process will
+attempt to compile all the other files, and will terminate with a
+non-zero status code.
+@end defun
+
+Native compilation can be run entirely asynchronously, in a
+sub-process of the main Emacs process.  This leaves the main Emacs
+process free to use while the compilation runs in the background.
+This is the method used by Emacs to natively-compile any Lisp file or
+byte-compiled Lisp file that is loaded into Emacs, when no
+natively-compiled file for it is available.
+
+@defun native-compile-async files &optional recursively load selector
+This function compiles the named @var{files} asynchronously.  The
+argument @var{files} should be a single file name (a string) or a list
+of one or more file and/or directory names.  If directories are
+present in the list, the optional argument @var{recursively} should be
+non-@code{nil} to cause the compilation to recurse into those
+directories.  If @var{load} is non-@code{nil}, Emacs will load each
+file that it succeeded to compile.  The optional argument
+@var{selector} allows control of which of @var{files} will be
+compiled; it can have one of the following values:
+
+@table @asis
+@item @code{nil} or omitted
+Select all the files and directories in @var{files}.
+@item a regular expression string
+Select the files and directories whose names match the regexp.
+@item a function
+A predicate function, which will be called with each file and
+directory in @var{files}, and should return non-@code{nil} if the file
+or the directory should be selected for compilation.
+@end table
+
+On systems with multiple CPU execution units, when @var{files} names
+more than one file, this function will normally start several
+compilation sub-processes in parallel, under the control of
+@code{native-comp-async-jobs-number} (@pxref{Native-Compilation
+Variables}).
+@end defun
+
+  The following function allows Lisp program to test whether
+native-compilation is available at runtime.
+
+@defun native-comp-available-p
+This function returns non-@code{nil} if the running Emacs process has
+the native-compilation support compiled into it.  On systems that load
+@file{libgccjit} dynamically, it also makes sure that library is
+available and can be loaded.  Lisp programs that need to know up front
+whether native-compilation is available should use this predicate.
+@end defun
+
+@node Native-Compilation Variables
+@section Native-Compilation Variables
+@cindex native-compilation variable
+
+  This section documents the variables that control
+native-compilation.
+
+@defopt native-comp-speed
+This variable specifies the optimization level for native compilation.
+Its value should be a number between @minus{}1 and 3.  Values between
+0 and 3 specify the optimization levels equivalent to the
+corresponding compiler @option{-O0}, @option{-O1}, etc.@: command-line
+options of the compiler.  The value @minus{}1 means disable
+native-compilation; functions and files will be only byte-compiled.
+The default value is 2.
+@end defopt
+
+@defopt native-comp-debug
+This variable specifies the level of debugging information produced by
+native-compilation.  Its value should be a number between zero and 3,
+with the following meaning:
+
+@table @asis
+@item 0
+No debugging output.  This is the default.
+@item 1
+Emit debugging symbols with the native code.  This allows easier
+debugging of the native code with debuggers such as @command{gdb}.
+@item 2
+Like 1, and in addition dump pseudo-C code.
+@item 3
+Like 2, and in addition dump the GCC intermediate passes and
+@file{libgccjit} log file.
+@end table
+@end defopt
+
+@defopt native-comp-verbose
+This variable controls the verbosity of native-compilation by
+suppressing some or all of the log messages emitted by it.  If its
+value is zero, the default, all of the log messages are suppressed.
+Setting it to a value between 1 and 3 will allow logging of the
+messages whose level is above the value.  The values have the
+following interpretations:
+
+@table @asis
+@item 0
+No logging.  This is the default.
+@item 1
+Log the final @acronym{LIMPLE} representation of the code.
+@item 2
+Log the @acronym{LAP}, the final @acronym{LIMPLE}, and some additional
+pass info.
+@item 3
+Maximum verbosity: log everything.
+@end table
+@end defopt
+
+@defopt native-comp-async-jobs-number
+This variable determines the maximum number of native-compilation
+subprocesses that will be started simultaneously.  It should be a
+non-negative number.  The default value is zero, which means use half
+the number of the CPU execution units, or 1 if the CPU has only one
+execution unit.
+@end defopt
+
+@defopt native-comp-async-report-warnings-errors
+If this variable's value is non-@code{nil}, warnings and errors from
+asynchronous native-compilation subprocesses are reported in the main
+Emacs session.  The default is @code{t}.
+@end defopt
+
+@defopt native-comp-async-query-on-exit
+If this variable's value is non-nil, Emacs will query upon exiting
+whether to exit and kill any asynchronous native-compilation
+subprocesses that are still running, thus preventing the corresponding
+@file{.eln} files from being written.  If the value is @code{nil}, the
+default, Emacs will kill these subprocesses without querying.
+@end defopt
diff --git a/doc/lispref/elisp.texi b/doc/lispref/elisp.texi
index be0c835..8b440c7 100644
--- a/doc/lispref/elisp.texi
+++ b/doc/lispref/elisp.texi
@@ -197,6 +197,7 @@ To view this manual in other formats, click
 
 * Loading::                 Reading files of Lisp code into Lisp.
 * Byte Compilation::        Compilation makes programs run faster.
+* Native Compilation::      Compile Lisp into native machine code.
 * Debugging::               Tools and tips for debugging Lisp programs.
 
 * Read and Print::          Converting Lisp objects to text and back.
@@ -646,6 +647,11 @@ Byte Compilation
 * Byte-Code Objects::       The data type used for byte-compiled functions.
 * Disassembly::             Disassembling byte-code; how to read byte-code.
 
+Native Compilation
+
+* Native-Compilation Functions::  Functions to natively-compile Lisp.
+* Native-Compilation Variables::  Variables controlling native compilation.
+
 Debugging Lisp Programs
 
 * Debugger::                A debugger for the Emacs Lisp evaluator.
diff --git a/doc/lispref/loading.texi b/doc/lispref/loading.texi
index e68a1ef..4d683da 100644
--- a/doc/lispref/loading.texi
+++ b/doc/lispref/loading.texi
@@ -71,7 +71,11 @@ forms in it, and closes the file.
 To find the file, @code{load} first looks for a file named
 @file{@var{filename}.elc}, that is, for a file whose name is
 @var{filename} with the extension @samp{.elc} appended.  If such a
-file exists, it is loaded.  If there is no file by that name, then
+file exists, and Emacs was compiled with native-compilation support
+(@pxref{Native Compilation}), @code{load} attempts to find a
+corresponding @samp{.eln} file, and if found, loads it instead of
+@file{@var{filename}.elc}.  Otherwise, it loads
+@file{@var{filename}.elc}.  If there is no file by that name, then
 @code{load} looks for a file named @file{@var{filename}.el}.  If that
 file exists, it is loaded.  If Emacs was compiled with support for
 dynamic modules (@pxref{Dynamic Modules}), @code{load} next looks for
@@ -109,6 +113,8 @@ explicit directory name.
 If the option @code{load-prefer-newer} is non-@code{nil}, then when
 searching suffixes, @code{load} selects whichever version of a file
 (@samp{.elc}, @samp{.el}, etc.)@: has been modified most recently.
+In this case, @code{load} doesn't load the @samp{.eln}
+natively-compiled file even if it exists.
 
 If @var{filename} is a relative file name, such as @file{foo} or
 @file{baz/foo.bar}, @code{load} searches for the file using the variable
@@ -153,7 +159,8 @@ during compilation.  @xref{Compiling Macros}.
 
 Messages like @samp{Loading foo...} and @samp{Loading foo...done} appear
 in the echo area during loading unless @var{nomessage} is
-non-@code{nil}.
+non-@code{nil}.  If a natively-compiled @samp{.eln} file is loaded,
+the message says so.
 
 @cindex load errors
 Any unhandled errors while loading a file terminate loading.  If the
@@ -430,6 +437,28 @@ optional argument @code{stringp} is non-@code{nil}, it 
instead returns
 the shadowed files as a string.
 @end deffn
 
+  If Emacs was compiled with support for native compilation
+(@pxref{Native Compilation}), then when a @samp{.elc} byte-compiled
+file is found by searching @code{load-path}, Emacs will try to look
+for a corresponding @samp{.eln} file holding the corresponding
+natively-compiled code.  The natively-compiled files are looked up in
+the directories listed by the @code{native-comp-eln-load-path}.
+
+@vindex comp-native-version-dir
+@defvar native-comp-eln-load-path
+This variable holds a list of directories where Emacs looks for
+natively-compiled @samp{.eln} files.  File names in the list that are
+not absolute are interpreted as relative to @code{invocation-directory}
+(@pxref{System Environment}).  The last directory in the list is the
+system directory, i.e.@: the directory with @samp{.eln} files
+installed by the Emacs build and installation procedure.  In each of
+the directories in the list, Emacs looks for @samp{.eln} files in a
+subdirectory whose name is constructed from the Emacs version and an
+8-character hash that depends on the current native-compilation
+@acronym{ABI}; the name of this subdirectory is stored in the variable
+@code{comp-native-version-dir}.
+@end defvar
+
 @node Loading Non-ASCII
 @section Loading Non-@acronym{ASCII} Characters
 @cindex loading, and non-ASCII characters
diff --git a/etc/NEWS b/etc/NEWS
index 9f4ded3..6efadfe 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -26,7 +26,10 @@ applies, and please also update docstrings as needed.
 
 ** Emacs now optionally supports native compilation of Lisp files.
 To enable this, configure Emacs with the '--with-native-compilation' option.
-This requires the libgccjit library to be installed and functional.
+This requires the libgccjit library to be installed and functional,
+and also requires GCC and Binutils to be available when Lisp code is
+natively compiled.  See the Info node "(elisp) Native Compilation" for
+more details.
 
 ---
 ** Support for building with Motif has been removed.