Re: LLVM/JIT on MacOS X

[Ben Abbott]

On Jun 12, 2014, at 1:36 PM, Ben Abbott <address@hidden> wrote:

> On Jun 12, 2014, at 11:36 AM, Stefan Mahr <address@hidden> wrote:
> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Ben Abbott wrote: 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Carlo,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Have you been successful getting JIT to work on Mac OS X?  When I 
>>>>>>>>>>>>> try, I encounter the error below.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Ben
>>>>>>>>>>>>> 
>>>>>>>>>>>>> [...]
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Undefined symbols for architecture x86_64:
>>>>>>>>>>>>> "llvm::verifyModule(llvm::Module const&, 
>>>>>>>>>>>>> llvm::VerifierFailureAction, std::basic_string<char, 
>>>>>>>>>>>>> std::char_traits<char>, std::allocator<char> >*)", referenced 
>>>>>>>>>>>>> from:
>>>>>>>>>>>>> tree_jit::optimize(llvm::Function*)      in 
>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>> "llvm::raw_fd_ostream::raw_fd_ostream(char const*, 
>>>>>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
>>>>>>>>>>>>> std::allocator<char> >&, unsigned int)", referenced from:
>>>>>>>>>>>>> tree_jit::optimize(llvm::Function*)      in 
>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>> "llvm::ExecutionEngine::createJIT(llvm::Module*, 
>>>>>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
>>>>>>>>>>>>> std::allocator<char> >*, llvm::JITMemoryManager*, 
>>>>>>>>>>>>> llvm::CodeGenOpt::Level, bool, llvm::Reloc::Model, 
>>>>>>>>>>>>> llvm::CodeModel::Model)", referenced from:
>>>>>>>>>>>>> tree_jit::initialize()      in 
>>>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>>>> ld: symbol(s) not found for architecture x86_64
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi Ben,
>>>>>>>>>>>> 
>>>>>>>>>>>> your error message sounds like a incompatibility with newer LLVM 
>>>>>>>>>>>> versions and should be fixed with my attached patch in this bug 
>>>>>>>>>>>> report: http://savannah.gnu.org/bugs/?41061
>>>>>>>>>>>> 
>>>>>>>>>>>> As always, a rebased changeset to a more recent repository state 
>>>>>>>>>>>> can be found here: 
>>>>>>>>>>>> http://inversethought.com/hg/octave-nkf/nkf-ready
>>>>>>>>>>>> 
>>>>>>>>>>>> Could you try the patch?
>>>>>>>>>>>> 
>>>>>>>>>>>> Stefan
>>>>>>>>>>> 
>>>>>>>>>>> Patching my default branch with the changeset from bug 41061 ...
>>>>>>>>>>> 
>>>>>>>>>>> $ patch -p1 < ~/Downloads/llvm-3.4-3.5pre.patch 
>>>>>>>>>>> patching file configure.ac
>>>>>>>>>>> patching file libinterp/corefcn/jit-typeinfo.cc
>>>>>>>>>>> patching file libinterp/corefcn/jit-util.h
>>>>>>>>>>> patching file libinterp/corefcn/pt-jit.cc
>>>>>>>>>>> Hunk #2 succeeded at 2064 (offset -8 lines).
>>>>>>>>>>> Hunk #3 succeeded at 2187 (offset -8 lines).
>>>>>>>>>>> patching file libinterp/corefcn/pt-jit.h
>>>>>>>>>>> patching file m4/acinclude.m4
>>>>>>>>>>> $ make -j4
>>>>>>>>>>> <snip>
>>>>>>>>>>> </snip>
>>>>>>>>>>> Undefined symbols for architecture x86_64:
>>>>>>>>>>> "llvm::verifyModule(llvm::Module const&, 
>>>>>>>>>>> llvm::VerifierFailureAction, std::basic_string<char, 
>>>>>>>>>>> std::char_traits<char>, std::allocator<char> >*)", referenced from:
>>>>>>>>>>> tree_jit::optimize(llvm::Function*)      in 
>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>> "llvm::raw_fd_ostream::raw_fd_ostream(char const*, 
>>>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
>>>>>>>>>>> std::allocator<char> >&, unsigned int)", referenced from:
>>>>>>>>>>> tree_jit::optimize(llvm::Function*)      in 
>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>> "llvm::ExecutionEngine::createJIT(llvm::Module*, 
>>>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
>>>>>>>>>>> std::allocator<char> >*, llvm::JITMemoryManager*, 
>>>>>>>>>>> llvm::CodeGenOpt::Level, bool, llvm::Reloc::Model, 
>>>>>>>>>>> llvm::CodeModel::Model)", referenced from:
>>>>>>>>>>> tree_jit::initialize()      in 
>>>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
>>>>>>>>>>> ld: symbol(s) not found for architecture x86_64
>>>>>>>>>>> 
>>>>>>>>>>> Ben
>>>>>>>>>> 
>>>>>>>>>> bootstrap and configure is needed before starting make.
>>>>>>>>> 
>>>>>>>>> Configure ran (automatically) after the patch was applied.  Just to 
>>>>>>>>> be sure, I ran bootstrap & configure manually, and still obtained the 
>>>>>>>>> same result.
>>>>>>>>> 
>>>>>>>>> Ben
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> I'm really confused that compiling works, but linking fails. This 
>>>>>>>> usually happens in the past (for me) if there are multiple versions of 
>>>>>>>> LLVM installed (Debian/Ubuntu) or some older headerfiles were not 
>>>>>>>> deleted during update (seen with MXE).
>>>>>>>> 
>>>>>>>> E.g. for linking error with llvm::verifyModule. The header file for my 
>>>>>>>> LLVM-3.3 installation is in <llvm/Analysis/Verifier.h>. For llvm 3.5 
>>>>>>>> it's in <llvm/IR/Verifier.h> with different declaration. If compiling 
>>>>>>>> with 3.3 headers and linking to 3.5 lib I would expect the same error 
>>>>>>>> as you get. Same for llvm::raw_fd_ostream::raw_fd_ostream, since the 
>>>>>>>> last argument is not an integer type anymore in newer LLVM versions. 
>>>>>>>> However, this doesn't explain a link error for 
>>>>>>>> llvm::ExecutionEngine::createJIT. I see no difference in declarions 
>>>>>>>> between LLVM versions 3.3 to 3.5.
>>>>>>>> 
>>>>>>>> To summarise: If you only have LLVM 3.3 installed, I have no idea. 
>>>>>>>> Sorry.
>>>>>>>> 
>>>>>>>> Stefan
>>>>>>> 
>>>>>>> hmmm, it does appear there is more than one llvm present.
>>>>>>> 
>>>>>>> $ port installed *llvm*
>>>>>>> The following ports are currently installed:
>>>>>>> llvm-3.3 @3.3_1
>>>>>>> llvm-3.3 @3.3_4 (active)
>>>>>>> llvm-gcc42 @2336.11_1 (active)
>>>>>>> llvm_select @0.2_0
>>>>>>> llvm_select @1.0_0 (active)
>>>>>>> 
>>>>>>> I've deactivate llvm-gcc42
>>>>>>> 
>>>>>>> $ sudo port deactivate llvm-gcc42
>>>>>>> Password:
>>>>>>> --->  Deactivating llvm-gcc42 @2336.11_1
>>>>>>> --->  Cleaning llvm-gcc42
>>>>>>> 
>>>>>>> And ran configure/make again. Unfortunately, I'm still seeing the same 
>>>>>>> error.
>>>>>>> 
>>>>>>> Looking at "config.log", there is one problem with llvm ....
>>>>>>> 
>>>>>>> configure:15039: checking llvm/IR/Function.h presence
>>>>>>> configure:15039: /opt/local/bin/g++-mp-4.7 -E -D__STDC_CONSTANT_MACROS 
>>>>>>> -D__STDC_LIMIT_MACROS -isystem /opt/local/libexec/llvm-3.3/include 
>>>>>>> -D_THREAD_SAFE -I/opt/local/include conftest.cpp
>>>>>>> configure:15039: $? = 0
>>>>>>> configure:15039: result: yes
>>>>>>> configure:15039: checking for llvm/IR/Function.h
>>>>>>> configure:15039: result: yes
>>>>>>> configure:15057: checking llvm/Support/IRBuilder.h usability
>>>>>>> configure:15057: /opt/local/bin/g++-mp-4.7 -c  -pipe -O2 -m64 
>>>>>>> -D_THREAD_SAFE -pthread -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS 
>>>>>>> -isystem /opt/local/libexec/llvm-3.3/include -D_THREAD_SAFE 
>>>>>>> -I/opt/local/include conftest.cpp >&5
>>>>>>> conftest.cpp:84:36: fatal error: llvm/Support/IRBuilder.h: No such file 
>>>>>>> or directory
>>>>>>> 
>>>>>>> Ben
>>>>>> 
>>>>>> The output of config.log looks OK. configure checks for presence of
>>>>>> llvm/Support/IRBuilder.h, but with LLVM 3.3 the header file is located
>>>>>> in llvm/IR/IRBuilder.h.
>>>>> 
>>>>> hmmm ... I checked config.log a second time ... looks like I examined the 
>>>>> wrong file previously. 
>>>>> 
>>>>> configure:15075: checking llvm/Target/TargetData.h usability
>>>>> configure:15075: /opt/local/bin/g++-mp-4.7 -c  -pipe -O2 -m64 
>>>>> -D_THREAD_SAFE -pthread -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS 
>>>>> -isystem /opt/local/libexec/llvm-3.3/include -D_THREAD_SAFE 
>>>>> -I/opt/local/include conftest.cpp >&5
>>>>> conftest.cpp:85:36: fatal error: llvm/Target/TargetData.h: No such file 
>>>>> or directory
>>>>> 
>>>>> I checked my install and have confirmed that TargetData.h does not exist.
>>>>> 
>>>>> Ben
>>>> 
>>>> No, same as before. For LLVM 3.3 the correct header file is 
>>>> llvm/IR/Datalayout.h, so the configure check is absolutely right.
>>> 
>>> Do you mean that I don't need llvm/Target/TargetData.h because I have 
>>> llvm/IR/Datalayout.h?
>>> 
>>> Ben
>>> 
>> 
>> Yes, unfortunately the LLVM guys love to move and rename files and change 
>> the API.
>> 
>> The include for llvm/IR/Datalayout.h is in pt-jit.cc:
>> 
>> #ifdef HAVE_LLVM_IR_DATALAYOUT_H
>> #include <llvm/IR/DataLayout.h>
>> #elif defined(HAVE_LLVM_DATALAYOUT_H)
>> #include <llvm/DataLayout.h>
>> #else
>> #include <llvm/Target/TargetData.h>
>> #endif
>> 
>> So you should have only one of this files in your include path.
>> 
>> Did you already check if MacOS provides it's own version of the llvm 
>> library, independant from the one of macports?
> 
> Yes. MacOS does have its own llvm.
> 
> $ ls -l /usr/bin/ll*
> -rwxr-xr-x  1 root  wheel  14224 Jun  2 23:52 /usr/bin/lldb
> lrwxr-xr-x  1 root  wheel      7 Oct 22  2013 /usr/bin/llvm-g++ -> clang++
> lrwxr-xr-x  1 root  wheel      5 Oct 22  2013 /usr/bin/llvm-gcc -> clang
> 
> But, I assume the llcm libs are part of the clang libs.
> 
> Ben

If anyone has a suggestion for resolving/debugging the error below ...

libtool: link: /opt/local/bin/g++-mp-4.7 -dynamiclib  -o 
.libs/liboctinterp.2.dylib  .libs/liboctinterp_la-octave.o 
.libs/liboctinterp_la-version.o operators/.libs/liboctinterp_la-op-b-b.o 
operators/.libs/liboctinterp_la-op-b-bm.o 
operators/.libs/liboctinterp_la-op-b-sbm.o 
operators/.libs/liboctinterp_la-op-bm-b.o 
operators/.libs/liboctinterp_la-op-bm-bm.o 
operators/.libs/liboctinterp_la-op-bm-sbm.o 
operators/.libs/liboctinterp_la-op-cdm-cdm.o 
operators/.libs/liboctinterp_la-op-cdm-cm.o 
operators/.libs/liboctinterp_la-op-cdm-cs.o 
operators/.libs/liboctinterp_la-op-cdm-dm.o 
operators/.libs/liboctinterp_la-op-cdm-m.o 
operators/.libs/liboctinterp_la-op-cdm-s.o 
operators/.libs/liboctinterp_la-op-cell.o 
operators/.libs/liboctinterp_la-op-chm.o 
operators/.libs/liboctinterp_la-op-class.o 
operators/.libs/liboctinterp_la-op-cm-cdm.o 
operators/.libs/liboctinterp_la-op-cm-cm.o 
operators/.libs/liboctinterp_la-op-cm-cs.o 
operators/.libs/liboctinterp_la-op-cm-dm.o 
operators/.libs/liboctinterp_la-op-cm-m.o 
operators/.libs/liboctinterp_la-op-cm-pm.o 
operators/.libs/liboctinterp_la-op-cm-s.o 
operators/.libs/liboctinterp_la-op-cm-scm.o 
operators/.libs/liboctinterp_la-op-cm-sm.o 
operators/.libs/liboctinterp_la-op-cs-cm.o 
operators/.libs/liboctinterp_la-op-cs-cs.o 
operators/.libs/liboctinterp_la-op-cs-m.o 
operators/.libs/liboctinterp_la-op-cs-s.o 
operators/.libs/liboctinterp_la-op-cs-scm.o 
operators/.libs/liboctinterp_la-op-cs-sm.o 
operators/.libs/liboctinterp_la-op-dm-cdm.o 
operators/.libs/liboctinterp_la-op-dm-cm.o 
operators/.libs/liboctinterp_la-op-dm-cs.o 
operators/.libs/liboctinterp_la-op-dm-dm.o 
operators/.libs/liboctinterp_la-op-dm-m.o 
operators/.libs/liboctinterp_la-op-dm-s.o 
operators/.libs/liboctinterp_la-op-dm-scm.o 
operators/.libs/liboctinterp_la-op-dm-sm.o 
operators/.libs/liboctinterp_la-op-double-conv.o 
operators/.libs/liboctinterp_la-op-fcdm-fcdm.o 
operators/.libs/liboctinterp_la-op-fcdm-fcm.o 
operators/.libs/liboctinterp_la-op-fcdm-fcs.o 
operators/.libs/liboctinterp_la-op-fcdm-fdm.o 
operators/.libs/liboctinterp_la-op-fcdm-fm.o 
operators/.libs/liboctinterp_la-op-fcdm-fs.o 
operators/.libs/liboctinterp_la-op-fcm-fcdm.o 
operators/.libs/liboctinterp_la-op-fcm-fcm.o 
operators/.libs/liboctinterp_la-op-fcm-fcs.o 
operators/.libs/liboctinterp_la-op-fcm-fdm.o 
operators/.libs/liboctinterp_la-op-fcm-fm.o 
operators/.libs/liboctinterp_la-op-fcm-fs.o 
operators/.libs/liboctinterp_la-op-fcm-pm.o 
operators/.libs/liboctinterp_la-op-fcn.o 
operators/.libs/liboctinterp_la-op-fcs-fcm.o 
operators/.libs/liboctinterp_la-op-fcs-fcs.o 
operators/.libs/liboctinterp_la-op-fcs-fm.o 
operators/.libs/liboctinterp_la-op-fcs-fs.o 
operators/.libs/liboctinterp_la-op-fdm-fcdm.o 
operators/.libs/liboctinterp_la-op-fdm-fcm.o 
operators/.libs/liboctinterp_la-op-fdm-fcs.o 
operators/.libs/liboctinterp_la-op-fdm-fdm.o 
operators/.libs/liboctinterp_la-op-fdm-fm.o 
operators/.libs/liboctinterp_la-op-fdm-fs.o 
operators/.libs/liboctinterp_la-op-float-conv.o 
operators/.libs/liboctinterp_la-op-fm-fcdm.o 
operators/.libs/liboctinterp_la-op-fm-fcm.o 
operators/.libs/liboctinterp_la-op-fm-fcs.o 
operators/.libs/liboctinterp_la-op-fm-fdm.o 
operators/.libs/liboctinterp_la-op-fm-fm.o 
operators/.libs/liboctinterp_la-op-fm-fs.o 
operators/.libs/liboctinterp_la-op-fm-pm.o 
operators/.libs/liboctinterp_la-op-fs-fcm.o 
operators/.libs/liboctinterp_la-op-fs-fcs.o 
operators/.libs/liboctinterp_la-op-fs-fm.o 
operators/.libs/liboctinterp_la-op-fs-fs.o 
operators/.libs/liboctinterp_la-op-i16-i16.o 
operators/.libs/liboctinterp_la-op-i32-i32.o 
operators/.libs/liboctinterp_la-op-i64-i64.o 
operators/.libs/liboctinterp_la-op-i8-i8.o 
operators/.libs/liboctinterp_la-op-int-concat.o 
operators/.libs/liboctinterp_la-op-int-conv.o 
operators/.libs/liboctinterp_la-op-m-cdm.o 
operators/.libs/liboctinterp_la-op-m-cm.o 
operators/.libs/liboctinterp_la-op-m-cs.o 
operators/.libs/liboctinterp_la-op-m-dm.o 
operators/.libs/liboctinterp_la-op-m-m.o 
operators/.libs/liboctinterp_la-op-m-pm.o 
operators/.libs/liboctinterp_la-op-m-s.o 
operators/.libs/liboctinterp_la-op-m-scm.o 
operators/.libs/liboctinterp_la-op-m-sm.o 
operators/.libs/liboctinterp_la-op-pm-cm.o 
operators/.libs/liboctinterp_la-op-pm-fcm.o 
operators/.libs/liboctinterp_la-op-pm-fm.o 
operators/.libs/liboctinterp_la-op-pm-m.o 
operators/.libs/liboctinterp_la-op-pm-pm.o 
operators/.libs/liboctinterp_la-op-pm-scm.o 
operators/.libs/liboctinterp_la-op-pm-sm.o 
operators/.libs/liboctinterp_la-op-range.o 
operators/.libs/liboctinterp_la-op-s-cm.o 
operators/.libs/liboctinterp_la-op-s-cs.o 
operators/.libs/liboctinterp_la-op-s-m.o 
operators/.libs/liboctinterp_la-op-s-s.o 
operators/.libs/liboctinterp_la-op-s-scm.o 
operators/.libs/liboctinterp_la-op-s-sm.o 
operators/.libs/liboctinterp_la-op-sbm-b.o 
operators/.libs/liboctinterp_la-op-sbm-bm.o 
operators/.libs/liboctinterp_la-op-sbm-sbm.o 
operators/.libs/liboctinterp_la-op-scm-cm.o 
operators/.libs/liboctinterp_la-op-scm-cs.o 
operators/.libs/liboctinterp_la-op-scm-m.o 
operators/.libs/liboctinterp_la-op-scm-s.o 
operators/.libs/liboctinterp_la-op-scm-scm.o 
operators/.libs/liboctinterp_la-op-scm-sm.o 
operators/.libs/liboctinterp_la-op-sm-cm.o 
operators/.libs/liboctinterp_la-op-sm-cs.o 
operators/.libs/liboctinterp_la-op-sm-m.o 
operators/.libs/liboctinterp_la-op-sm-s.o 
operators/.libs/liboctinterp_la-op-sm-scm.o 
operators/.libs/liboctinterp_la-op-sm-sm.o 
operators/.libs/liboctinterp_la-op-str-m.o 
operators/.libs/liboctinterp_la-op-str-s.o 
operators/.libs/liboctinterp_la-op-str-str.o 
operators/.libs/liboctinterp_la-op-struct.o 
operators/.libs/liboctinterp_la-op-ui16-ui16.o 
operators/.libs/liboctinterp_la-op-ui32-ui32.o 
operators/.libs/liboctinterp_la-op-ui64-ui64.o 
operators/.libs/liboctinterp_la-op-ui8-ui8.o 
template-inst/.libs/liboctinterp_la-Array-os.o 
template-inst/.libs/liboctinterp_la-Array-tc.o 
template-inst/.libs/liboctinterp_la-Array-jit.o 
corefcn/.libs/liboctinterp_la-oct-errno.o operators/.libs/liboctinterp_la-ops.o 
.libs/liboctinterp_la-builtins.o   
-Wl,-force_load,octave-value/.libs/liboctave-value.a 
-Wl,-force_load,parse-tree/.libs/libparse-tree.a 
-Wl,-force_load,parse-tree/.libs/libparser.a 
-Wl,-force_load,corefcn/.libs/libcorefcn.a 
-Wl,-force_load,corefcn/.libs/libtex_parser.a  
-L/opt/local/libexec/llvm-3.3/lib -L/opt/local/lib 
../liboctave/.libs/liboctave.dylib 
-L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin13/4.7.3 
-L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin13/4.7.3/../../.. -lhdf5 -lz 
-lfontconfig -lfreetype -lgl2ps -lLLVM-3.3 -lcurl -lcholmod -lumfpack 
-lsuitesparseconfig -lamd -lcamd -lcolamd -lccolamd -lcxsparse -larpack 
-lqrupdate -lfftw3_threads -lfftw3 -lfftw3f_threads -lfftw3f -llapack -lcblas 
-lf77blas -latlas -lreadline -lncurses -lpcre -ldl -lgfortran -lquadmath -lm  
-O2 -m64 -pthread -m64 -Wl,-framework -Wl,OpenGL -Wl,-framework -Wl,Carbon   
-pthread -install_name  
/usr/local/octave/3.8.2/lib/octave/4.1.0+/liboctinterp.2.dylib 
-compatibility_version 3 -current_version 3.0 -Wl,-single_module
Undefined symbols for architecture x86_64:
  "llvm::verifyModule(llvm::Module const&, llvm::VerifierFailureAction, 
std::basic_string<char, std::char_traits<char>, std::allocator<char> >*)", 
referenced from:
      tree_jit::optimize(llvm::Function*)      in 
libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
  "llvm::raw_fd_ostream::raw_fd_ostream(char const*, std::basic_string<char, 
std::char_traits<char>, std::allocator<char> >&, unsigned int)", referenced 
from:
      tree_jit::optimize(llvm::Function*)      in 
libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
  "llvm::ExecutionEngine::createJIT(llvm::Module*, std::basic_string<char, 
std::char_traits<char>, std::allocator<char> >*, llvm::JITMemoryManager*, 
llvm::CodeGenOpt::Level, bool, llvm::Reloc::Model, llvm::CodeModel::Model)", 
referenced from:
      tree_jit::initialize()      in 
libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
ld: symbol(s) not found for architecture x86_64

... I'm happy to try.

Ben