Re: LLVM/JIT on MacOS X

[Stefan Mahr]

> On Jun 14, 2014, at 10:26 AM, Ben Abbott <address@hidden> wrote:
> 
>> On Jun 12, 2014, at 1:36 PM, Ben Abbott <address@hidden> wrote:
>>
>>> On Jun 12, 2014, at 11:36 AM, Stefan Mahr <address@hidden> wrote:
>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ben Abbott wrote: 
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Carlo,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Have you been successful getting JIT to work on Mac OS X?  When 
>>>>>>>>>>>>>>> I try, I encounter the error below.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ben
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [...]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Undefined symbols for architecture x86_64:
>>>>>>>>>>>>>>> "llvm::verifyModule(llvm::Module const&, 
>>>>>>>>>>>>>>> llvm::VerifierFailureAction, std::basic_string<char, 
>>>>>>>>>>>>>>> std::char_traits<char>, std::allocator<char> >*)", referenced 
>>>>>>>>>>>>>>> from:
>>>>>>>>>>>>>>> tree_jit::optimize(llvm::Function*)      in 
>>>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>>>> "llvm::raw_fd_ostream::raw_fd_ostream(char const*, 
>>>>>>>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
>>>>>>>>>>>>>>> std::allocator<char> >&, unsigned int)", referenced from:
>>>>>>>>>>>>>>> tree_jit::optimize(llvm::Function*)      in 
>>>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>>>> "llvm::ExecutionEngine::createJIT(llvm::Module*, 
>>>>>>>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
>>>>>>>>>>>>>>> std::allocator<char> >*, llvm::JITMemoryManager*, 
>>>>>>>>>>>>>>> llvm::CodeGenOpt::Level, bool, llvm::Reloc::Model, 
>>>>>>>>>>>>>>> llvm::CodeModel::Model)", referenced from:
>>>>>>>>>>>>>>> tree_jit::initialize()      in 
>>>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>>>> ld: symbol(s) not found for architecture x86_64
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Ben,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> your error message sounds like a incompatibility with newer LLVM 
>>>>>>>>>>>>>> versions and should be fixed with my attached patch in this bug 
>>>>>>>>>>>>>> report: http://savannah.gnu.org/bugs/?41061
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As always, a rebased changeset to a more recent repository state 
>>>>>>>>>>>>>> can be found here: 
>>>>>>>>>>>>>> http://inversethought.com/hg/octave-nkf/nkf-ready
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Could you try the patch?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Stefan
>>>>>>>>>>>>>
>>>>>>>>>>>>> Patching my default branch with the changeset from bug 41061 ...
>>>>>>>>>>>>>
>>>>>>>>>>>>> $ patch -p1 < ~/Downloads/llvm-3.4-3.5pre.patch 
>>>>>>>>>>>>> patching file configure.ac
>>>>>>>>>>>>> patching file libinterp/corefcn/jit-typeinfo.cc
>>>>>>>>>>>>> patching file libinterp/corefcn/jit-util.h
>>>>>>>>>>>>> patching file libinterp/corefcn/pt-jit.cc
>>>>>>>>>>>>> Hunk #2 succeeded at 2064 (offset -8 lines).
>>>>>>>>>>>>> Hunk #3 succeeded at 2187 (offset -8 lines).
>>>>>>>>>>>>> patching file libinterp/corefcn/pt-jit.h
>>>>>>>>>>>>> patching file m4/acinclude.m4
>>>>>>>>>>>>> $ make -j4
>>>>>>>>>>>>> <snip>
>>>>>>>>>>>>> </snip>
>>>>>>>>>>>>> Undefined symbols for architecture x86_64:
>>>>>>>>>>>>> "llvm::verifyModule(llvm::Module const&, 
>>>>>>>>>>>>> llvm::VerifierFailureAction, std::basic_string<char, 
>>>>>>>>>>>>> std::char_traits<char>, std::allocator<char> >*)", referenced 
>>>>>>>>>>>>> from:
>>>>>>>>>>>>> tree_jit::optimize(llvm::Function*)      in 
>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>> "llvm::raw_fd_ostream::raw_fd_ostream(char const*, 
>>>>>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
>>>>>>>>>>>>> std::allocator<char> >&, unsigned int)", referenced from:
>>>>>>>>>>>>> tree_jit::optimize(llvm::Function*)      in 
>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>> "llvm::ExecutionEngine::createJIT(llvm::Module*, 
>>>>>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
>>>>>>>>>>>>> std::allocator<char> >*, llvm::JITMemoryManager*, 
>>>>>>>>>>>>> llvm::CodeGenOpt::Level, bool, llvm::Reloc::Model, 
>>>>>>>>>>>>> llvm::CodeModel::Model)", referenced from:
>>>>>>>>>>>>> tree_jit::initialize()      in 
>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>> ld: symbol(s) not found for architecture x86_64
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ben
>>>>>>>>>>>>
>>>>>>>>>>>> bootstrap and configure is needed before starting make.
>>>>>>>>>>>
>>>>>>>>>>> Configure ran (automatically) after the patch was applied.  Just to 
>>>>>>>>>>> be sure, I ran bootstrap & configure manually, and still obtained 
>>>>>>>>>>> the same result.
>>>>>>>>>>>
>>>>>>>>>>> Ben
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'm really confused that compiling works, but linking fails. This 
>>>>>>>>>> usually happens in the past (for me) if there are multiple versions 
>>>>>>>>>> of LLVM installed (Debian/Ubuntu) or some older headerfiles were not 
>>>>>>>>>> deleted during update (seen with MXE).
>>>>>>>>>>
>>>>>>>>>> E.g. for linking error with llvm::verifyModule. The header file for 
>>>>>>>>>> my LLVM-3.3 installation is in <llvm/Analysis/Verifier.h>. For llvm 
>>>>>>>>>> 3.5 it's in <llvm/IR/Verifier.h> with different declaration. If 
>>>>>>>>>> compiling with 3.3 headers and linking to 3.5 lib I would expect the 
>>>>>>>>>> same error as you get. Same for 
>>>>>>>>>> llvm::raw_fd_ostream::raw_fd_ostream, since the last argument is not 
>>>>>>>>>> an integer type anymore in newer LLVM versions. However, this 
>>>>>>>>>> doesn't explain a link error for llvm::ExecutionEngine::createJIT. I 
>>>>>>>>>> see no difference in declarions between LLVM versions 3.3 to 3.5.
>>>>>>>>>>
>>>>>>>>>> To summarise: If you only have LLVM 3.3 installed, I have no idea. 
>>>>>>>>>> Sorry.
>>>>>>>>>>
>>>>>>>>>> Stefan
>>>>>>>>>
>>>>>>>>> hmmm, it does appear there is more than one llvm present.
>>>>>>>>>
>>>>>>>>> $ port installed *llvm*
>>>>>>>>> The following ports are currently installed:
>>>>>>>>> llvm-3.3 @3.3_1
>>>>>>>>> llvm-3.3 @3.3_4 (active)
>>>>>>>>> llvm-gcc42 @2336.11_1 (active)
>>>>>>>>> llvm_select @0.2_0
>>>>>>>>> llvm_select @1.0_0 (active)
>>>>>>>>>
>>>>>>>>> I've deactivate llvm-gcc42
>>>>>>>>>
>>>>>>>>> $ sudo port deactivate llvm-gcc42
>>>>>>>>> Password:
>>>>>>>>> --->  Deactivating llvm-gcc42 @2336.11_1
>>>>>>>>> --->  Cleaning llvm-gcc42
>>>>>>>>>
>>>>>>>>> And ran configure/make again. Unfortunately, I'm still seeing the 
>>>>>>>>> same error.
>>>>>>>>>
>>>>>>>>> Looking at "config.log", there is one problem with llvm ....
>>>>>>>>>
>>>>>>>>> configure:15039: checking llvm/IR/Function.h presence
>>>>>>>>> configure:15039: /opt/local/bin/g++-mp-4.7 -E 
>>>>>>>>> -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS -isystem 
>>>>>>>>> /opt/local/libexec/llvm-3.3/include -D_THREAD_SAFE 
>>>>>>>>> -I/opt/local/include conftest.cpp
>>>>>>>>> configure:15039: $? = 0
>>>>>>>>> configure:15039: result: yes
>>>>>>>>> configure:15039: checking for llvm/IR/Function.h
>>>>>>>>> configure:15039: result: yes
>>>>>>>>> configure:15057: checking llvm/Support/IRBuilder.h usability
>>>>>>>>> configure:15057: /opt/local/bin/g++-mp-4.7 -c  -pipe -O2 -m64 
>>>>>>>>> -D_THREAD_SAFE -pthread -D__STDC_CONSTANT_MACROS 
>>>>>>>>> -D__STDC_LIMIT_MACROS -isystem /opt/local/libexec/llvm-3.3/include 
>>>>>>>>> -D_THREAD_SAFE -I/opt/local/include conftest.cpp >&5
>>>>>>>>> conftest.cpp:84:36: fatal error: llvm/Support/IRBuilder.h: No such 
>>>>>>>>> file or directory
>>>>>>>>>
>>>>>>>>> Ben
>>>>>>>>
> 
> I've switched to llvm-3.4. Configure looks to detect it correctly.
> 
>   LLVM CPPFLAGS:               -isystem /opt/local/libexec/llvm-3.4/include
>   LLVM LDFLAGS:                -L/opt/local/libexec/llvm-3.4/lib
>   LLVM libraries:              -lLLVM-3.4
> 
> While make still fails, the error is different.
> 
> libtool: compile:  /opt/local/bin/g++-mp-4.7 -DHAVE_CONFIG_H -I. -I.. 
> -I../liboctave/cruft/misc -I../liboctave/array -I../liboctave/numeric 
> -I../liboctave/numeric -I../liboctave/operators -I../liboctave/operators 
> -I../liboctave/system -I../liboctave/util -I./octave-value -I./operators 
> -Iparse-tree -I./parse-tree -Icorefcn -I./corefcn -I../libgnu -I../libgnu 
> -I/opt/local/include -I/opt/local/include/freetype2 -I/opt/local/include 
> -I/opt/local/include/freetype2 -I/opt/local/include 
> -I/opt/local/include/libpng16 -I/opt/local/include/freetype2 
> -I/opt/local/include -I/opt/local/include/libpng16 -isystem 
> /opt/local/libexec/llvm-3.4/include -D_THREAD_SAFE -I/opt/local/include -Wall 
> -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings 
> -Wcast-align -Wcast-qual -pipe -O2 -m64 -D_THREAD_SAFE -pthread -MT 
> corefcn/corefcn_libcorefcn_la-pt-jit.lo -MD -MP -MF 
> corefcn/.deps/corefcn_libcorefcn_la-pt-jit.Tpo -c corefcn/pt-jit.cc  
> -fno-common -DPIC -o corefcn/.libs/corefcn_libcorefcn
 _
la-pt-jit.o
> In file included from corefcn/pt-jit.cc:57:0:
> /opt/local/libexec/llvm-3.4/include/llvm/PassManager.h:34:15: error: 
> 'PassManager' is already declared in this scope
> /opt/local/libexec/llvm-3.4/include/llvm/PassManager.h:35:15: error: 
> 'FunctionPassManager' is already declared in this scope
> corefcn/pt-jit.cc: In member function 'bool tree_jit::initialize()':
> corefcn/pt-jit.cc:2056:48: error: cannot convert 'llvm::legacy::PassManager*' 
> to 'llvm::PassManager*' in assignment
> corefcn/pt-jit.cc:2057:22: error: invalid use of incomplete type 'class 
> llvm::PassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:46:9: error: forward declaration of 'class 
> llvm::PassManager'
> corefcn/pt-jit.cc:2059:55: error: cannot convert 
> 'llvm::legacy::FunctionPassManager*' to 'llvm::FunctionPassManager*' in 
> assignment
> corefcn/pt-jit.cc:2061:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2065:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2066:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2067:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2068:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2069:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2070:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2071:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2072:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc: In member function 'void 
> tree_jit::optimize(llvm::Function*)':
> corefcn/pt-jit.cc:2168:22: error: invalid use of incomplete type 'class 
> llvm::PassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:46:9: error: forward declaration of 'class 
> llvm::PassManager'
> corefcn/pt-jit.cc:2169:15: error: invalid use of incomplete type 'class 
> llvm::FunctionPassManager'
> In file included from corefcn/jit-typeinfo.h:34:0,
>                  from corefcn/jit-ir.h:34,
>                  from corefcn/pt-jit.h:30,
>                  from corefcn/pt-jit.cc:36:
> corefcn/jit-util.h:45:9: error: forward declaration of 'class 
> llvm::FunctionPassManager'
> corefcn/pt-jit.cc:2175:34: error: 'F_Binary' is not a member of 
> 'llvm::raw_fd_ostream'
> 
> Ben
> 

Hi Ben,

have you still applied the LLVM patch? (Rik has applied the patch to
default.)

Stefan