mingw-cross-env-list
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Mingw-cross-env-list] Crosscompiling Lua rocks with MXE

From: Nagaev Boris
Subject: [Mingw-cross-env-list] Crosscompiling Lua rocks with MXE
Date: Sun, 11 Oct 2015 05:19:08 +0300 
Hi,

I want to share Lua and MXE related code. This work is unfinished.

Hundreds of Lua packages are distributed via LuaRocks [1]. LuaRocks is
the package manager for Lua. It allows you to create and install Lua
modules as self-contained packages called rocks. You can download and
install LuaRocks on Unix and Windows.

Lua rocks are similar to Ruby gems, Python eggs or JavaScript NPM.
Command "luarocks install <rock>" downloads a rock from luarocks.org
(or other luarocks server), compiles C files (modules) to shared
libraries and installs (copies) shared libraries and Lua files to the
directory where luarocks is installed. Installed rocks can be loaded
from Lua with function "require".

Example:
$ luarocks install lpeg
$ lua -e 'http = require "socket.http"; print(http.request("http://mxe.cc";))'
<!DOCTYPE html>
....

LuaRocks can used with make, cmake, custom or builtin back-ends [2].

I have ported LuaRocks and some popular rocks to MXE in my fork,
branch "luarocks" [3]. LuaRocks can now be used in the same way as
CMake or Make.

Detailed description of changes is below.

1. Package lua now installs native executable usr/bin/lua and
cross-compiled lua executable usr/<target>/bin/lua.exe. Native
executable is needed since LuaRocks is written in Lua. Cross-compiled
is needed to run Lua scripts loading cross-compiled lua modules.

2. Package luarocks was added. Luarocks was patched to support new
platform "mxe", inherited from platform "unix". It uses mix of system
tools (e.g., openssl, ln, mkdir), MXE build chain
(i686-w64-mingw32.shared-gcc, i686-w64-mingw32.shared-cmake) and some
Windows variables (e.g., "dll" extension for shared libraries). The
package is shared-only because Lua loads modules in runtime. It
creates prefixed luarocks tool in usr/bin. It also creates prefixed
wine+lua wrapper aware of locations of dll and lua files installed.
This script can be used to test modules in Linux as if running them in
Windows.

There was a difficult choice if "mxe" platform of luarocks is
inherited from "windows" or "unix" platform. I tried both and it is
less patching for "unix". For "windows" even build tools differ, while
for "unix" it builds without patching or with minor patching (as other
MXE packages).

LuaRocks can be used to install rocks. With ideal rock it works as follows:

$ i686-w64-mingw32.shared-luarocks install <rock>

Downloads rockspeck, downloads sources, verifies checksum (useless
thing, because checksum is compared to the value from rockspec file,
which itself is neither verified nor signed), builds and installs.

3. Package lpeg was added. LPeg is a pattern-matching library for Lua,
based on Parsing Expression Grammars (PEGs). The only patch needed for
lpeg was adding the rockspec, which is not distributed with LPeg on
LPeg's author site [4].

Test:
$ ./usr/bin/i686-w64-mingw32.shared-lua
Lua 5.3.1  Copyright (C) 1994-2015 Lua.org, PUC-Rio
> lpeg = require 'lpeg'
> print(((lpeg.R('AZ'))^1):match("TEXT"))
5


4. Package luasocket was added. LuaSocket is the network support for
the Lua language [5]. It is shipped with the rockspec file, but it
needs some patching. Patch 1 tells luarocks to use Windows specific
set of C files when building on MXE. File src/usocket_dummy.lua is
used to prevent compiling modules socket.unix and socket.serial which
are defined for "unix" and are inherited by "mxe" if it doesn't
redefine them. Unix versions of these modules can't be compiled
because they include sys/pool.h (from file usocket.c). Patch 2 updates
source for Lua 5.4 (rename function luaL_checkint to
luaL_checkinteger, as luaL_checkint was removed from Lua 5.3).

Test:

$ ./usr/bin/i686-w64-mingw32.shared-lua -e 'http = require
"socket.http"; print(http.request("http://mxe.cc";))' | head -1
<!DOCTYPE html>


5. Package llthreads2 was added. LLthreads stands for Low-Level
threads (pthreads or WIN32 threads) for Lua [6]. LLthreads2 is full
dropin replacement for llthreads library rewritten without
`LuaNativeObjects` code generator [7]. The patch tells luarocks to
link against kernel32 instead of pthread when on MXE. The rockspec
already had two scenarios: pthread for unix and kernel32 for windows.
As "MXE" platform is inherited from "Unix", it would get "pthread"
linking by default.

Test:
$ ./usr/bin/i686-w64-mingw32.shared-lua
> ll = require 'llthreads2'
> t = ll.new[[print(123)]]
> t:start()
userdata: 00123da0
123
> t:start()
nil     Thread already started.
> t:join()
true
> t:join()
nil     Can't join a thread that has already been joined.

Note that I don't use luarocks to download source tarballs (as I said
above, it doesn't verify checksums properly) using MXE's downloading
and verifying facilities instead. Luarocks is used as builder,
installer and Lua library (it installs Lua files to
usr/i686-w64-mingw32.shared/share/lua/5.3/luarocks/).


What's next?

  * make your suggestions
  * maybe move lua rocks to stage/phase "lua". See [8] [9] [10]
  * rocks using cmake as a backend, e.g. lua-rapidjson [11]
  * push some patches to upstreams (lpeg, luasocket, llthreads)


[1] https://luarocks.org/
https://github.com/keplerproject/luarocks/wiki
[2] 
https://github.com/keplerproject/luarocks/wiki/Rockspec-format#Build_backends
[3] https://github.com/LuaAndC/mxe/commits/luarocks
[4] http://www.inf.puc-rio.br/~roberto/lpeg/
[5] https://github.com/diegonehab/luasocket
[6] https://github.com/Neopallium/lua-llthreads
[7] https://github.com/moteus/lua-llthreads2
[8] https://github.com/mxe/mxe/issues/881
[9] 
https://lists.nongnu.org/archive/html/mingw-cross-env-list/2015-09/msg00026.html
[10] https://github.com/mxe/mxe/issues/919
[11] https://github.com/xpol/lua-rapidjson

-- 


Best regards,
Boris Nagaev
reply via email to
[Prev in Thread] Current Thread [Next in Thread]