Aw: Re: LLVM/JIT on MacOS X

[Stefan Mahr]

> >>>>>>>>> 
> >>>>>>>>> Ben Abbott wrote: 
> >>>>>>>>>> 
> >>>>>>>>>> Carlo,
> >>>>>>>>>> 
> >>>>>>>>>> Have you been successful getting JIT to work on Mac OS X?  When I 
> >>>>>>>>>> try, I encounter the error below.
> >>>>>>>>>> 
> >>>>>>>>>> Ben
> >>>>>>>>>> 
> >>>>>>>>>> [...]
> >>>>>>>>>> 
> >>>>>>>>>> Undefined symbols for architecture x86_64:
> >>>>>>>>>> "llvm::verifyModule(llvm::Module const&, 
> >>>>>>>>>> llvm::VerifierFailureAction, std::basic_string<char, 
> >>>>>>>>>> std::char_traits<char>, std::allocator<char> >*)", referenced from:
> >>>>>>>>>>  tree_jit::optimize(llvm::Function*)      in 
> >>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
> >>>>>>>>>> "llvm::raw_fd_ostream::raw_fd_ostream(char const*, 
> >>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
> >>>>>>>>>> std::allocator<char> >&, unsigned int)", referenced from:
> >>>>>>>>>>  tree_jit::optimize(llvm::Function*)      in 
> >>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
> >>>>>>>>>> "llvm::ExecutionEngine::createJIT(llvm::Module*, 
> >>>>>>>>>> std::basic_string<char, std::char_traits<char>, 
> >>>>>>>>>> std::allocator<char> >*, llvm::JITMemoryManager*, 
> >>>>>>>>>> llvm::CodeGenOpt::Level, bool, llvm::Reloc::Model, 
> >>>>>>>>>> llvm::CodeModel::Model)", referenced from:
> >>>>>>>>>>  tree_jit::initialize()      in 
> >>>>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
> >>>>>>>>>> ld: symbol(s) not found for architecture x86_64
> >>>>>>>>>> 
> >>>>>>>>> 
> >>>>>>>>> Hi Ben,
> >>>>>>>>> 
> >>>>>>>>> your error message sounds like a incompatibility with newer LLVM 
> >>>>>>>>> versions and should be fixed with my attached patch in this bug 
> >>>>>>>>> report: http://savannah.gnu.org/bugs/?41061
> >>>>>>>>> 
> >>>>>>>>> As always, a rebased changeset to a more recent repository state 
> >>>>>>>>> can be found here: http://inversethought.com/hg/octave-nkf/nkf-ready
> >>>>>>>>> 
> >>>>>>>>> Could you try the patch?
> >>>>>>>>> 
> >>>>>>>>> Stefan
> >>>>>>>> 
> >>>>>>>> Patching my default branch with the changeset from bug 41061 ...
> >>>>>>>> 
> >>>>>>>> $ patch -p1 < ~/Downloads/llvm-3.4-3.5pre.patch 
> >>>>>>>> patching file configure.ac
> >>>>>>>> patching file libinterp/corefcn/jit-typeinfo.cc
> >>>>>>>> patching file libinterp/corefcn/jit-util.h
> >>>>>>>> patching file libinterp/corefcn/pt-jit.cc
> >>>>>>>> Hunk #2 succeeded at 2064 (offset -8 lines).
> >>>>>>>> Hunk #3 succeeded at 2187 (offset -8 lines).
> >>>>>>>> patching file libinterp/corefcn/pt-jit.h
> >>>>>>>> patching file m4/acinclude.m4
> >>>>>>>> $ make -j4
> >>>>>>>> <snip>
> >>>>>>>> </snip>
> >>>>>>>> Undefined symbols for architecture x86_64:
> >>>>>>>> "llvm::verifyModule(llvm::Module const&, 
> >>>>>>>> llvm::VerifierFailureAction, std::basic_string<char, 
> >>>>>>>> std::char_traits<char>, std::allocator<char> >*)", referenced from:
> >>>>>>>>   tree_jit::optimize(llvm::Function*)      in 
> >>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
> >>>>>>>> "llvm::raw_fd_ostream::raw_fd_ostream(char const*, 
> >>>>>>>> std::basic_string<char, std::char_traits<char>, std::allocator<char> 
> >>>>>>>> >&, unsigned int)", referenced from:
> >>>>>>>>   tree_jit::optimize(llvm::Function*)      in 
> >>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
> >>>>>>>> "llvm::ExecutionEngine::createJIT(llvm::Module*, 
> >>>>>>>> std::basic_string<char, std::char_traits<char>, std::allocator<char> 
> >>>>>>>> >*, llvm::JITMemoryManager*, llvm::CodeGenOpt::Level, bool, 
> >>>>>>>> llvm::Reloc::Model, llvm::CodeModel::Model)", referenced from:
> >>>>>>>>   tree_jit::initialize()      in 
> >>>>>>>> libcorefcn.a(corefcn_libcorefcn_la-pt-jit.o)
> >>>>>>>> ld: symbol(s) not found for architecture x86_64
> >>>>>>>> 
> >>>>>>>> Ben
> >>>>>>> 
> >>>>>>> bootstrap and configure is needed before starting make.
> >>>>>> 
> >>>>>> Configure ran (automatically) after the patch was applied.  Just to be 
> >>>>>> sure, I ran bootstrap & configure manually, and still obtained the 
> >>>>>> same result.
> >>>>>> 
> >>>>>> Ben
> >>>>>> 
> >>>>> 
> >>>>> I'm really confused that compiling works, but linking fails. This 
> >>>>> usually happens in the past (for me) if there are multiple versions of 
> >>>>> LLVM installed (Debian/Ubuntu) or some older headerfiles were not 
> >>>>> deleted during update (seen with MXE).
> >>>>> 
> >>>>> E.g. for linking error with llvm::verifyModule. The header file for my 
> >>>>> LLVM-3.3 installation is in <llvm/Analysis/Verifier.h>. For llvm 3.5 
> >>>>> it's in <llvm/IR/Verifier.h> with different declaration. If compiling 
> >>>>> with 3.3 headers and linking to 3.5 lib I would expect the same error 
> >>>>> as you get. Same for llvm::raw_fd_ostream::raw_fd_ostream, since the 
> >>>>> last argument is not an integer type anymore in newer LLVM versions. 
> >>>>> However, this doesn't explain a link error for 
> >>>>> llvm::ExecutionEngine::createJIT. I see no difference in declarions 
> >>>>> between LLVM versions 3.3 to 3.5.
> >>>>> 
> >>>>> To summarise: If you only have LLVM 3.3 installed, I have no idea. 
> >>>>> Sorry.
> >>>>> 
> >>>>> Stefan
> >>>> 
> >>>> hmmm, it does appear there is more than one llvm present.
> >>>> 
> >>>> $ port installed *llvm*
> >>>> The following ports are currently installed:
> >>>> llvm-3.3 @3.3_1
> >>>> llvm-3.3 @3.3_4 (active)
> >>>> llvm-gcc42 @2336.11_1 (active)
> >>>> llvm_select @0.2_0
> >>>> llvm_select @1.0_0 (active)
> >>>> 
> >>>> I've deactivate llvm-gcc42
> >>>> 
> >>>> $ sudo port deactivate llvm-gcc42
> >>>> Password:
> >>>> --->  Deactivating llvm-gcc42 @2336.11_1
> >>>> --->  Cleaning llvm-gcc42
> >>>> 
> >>>> And ran configure/make again. Unfortunately, I'm still seeing the same 
> >>>> error.
> >>>> 
> >>>> Looking at "config.log", there is one problem with llvm ....
> >>>> 
> >>>> configure:15039: checking llvm/IR/Function.h presence
> >>>> configure:15039: /opt/local/bin/g++-mp-4.7 -E -D__STDC_CONSTANT_MACROS 
> >>>> -D__STDC_LIMIT_MACROS -isystem /opt/local/libexec/llvm-3.3/include 
> >>>> -D_THREAD_SAFE -I/opt/local/include conftest.cpp
> >>>> configure:15039: $? = 0
> >>>> configure:15039: result: yes
> >>>> configure:15039: checking for llvm/IR/Function.h
> >>>> configure:15039: result: yes
> >>>> configure:15057: checking llvm/Support/IRBuilder.h usability
> >>>> configure:15057: /opt/local/bin/g++-mp-4.7 -c  -pipe -O2 -m64 
> >>>> -D_THREAD_SAFE -pthread -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS 
> >>>> -isystem /opt/local/libexec/llvm-3.3/include -D_THREAD_SAFE 
> >>>> -I/opt/local/include conftest.cpp >&5
> >>>> conftest.cpp:84:36: fatal error: llvm/Support/IRBuilder.h: No such file 
> >>>> or directory
> >>>> 
> >>>> Ben
> >>> 
> >>> The output of config.log looks OK. configure checks for presence of
> >>> llvm/Support/IRBuilder.h, but with LLVM 3.3 the header file is located
> >>> in llvm/IR/IRBuilder.h.
> >> 
> >> hmmm ... I checked config.log a second time ... looks like I examined the 
> >> wrong file previously. 
> >> 
> >> configure:15075: checking llvm/Target/TargetData.h usability
> >> configure:15075: /opt/local/bin/g++-mp-4.7 -c  -pipe -O2 -m64 
> >> -D_THREAD_SAFE -pthread -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS 
> >> -isystem /opt/local/libexec/llvm-3.3/include -D_THREAD_SAFE 
> >> -I/opt/local/include conftest.cpp >&5
> >> conftest.cpp:85:36: fatal error: llvm/Target/TargetData.h: No such file or 
> >> directory
> >> 
> >> I checked my install and have confirmed that TargetData.h does not exist.
> >> 
> >> Ben
> > 
> > No, same as before. For LLVM 3.3 the correct header file is 
> > llvm/IR/Datalayout.h, so the configure check is absolutely right.
> 
> Do you mean that I don't need llvm/Target/TargetData.h because I have 
> llvm/IR/Datalayout.h?
> 
> Ben
> 

Yes, unfortunately the LLVM guys love to move and rename files and change the 
API.

The include for llvm/IR/Datalayout.h is in pt-jit.cc:

#ifdef HAVE_LLVM_IR_DATALAYOUT_H
#include <llvm/IR/DataLayout.h>
#elif defined(HAVE_LLVM_DATALAYOUT_H)
#include <llvm/DataLayout.h>
#else
#include <llvm/Target/TargetData.h>
#endif

So you should have only one of this files in your include path.

Did you already check if MacOS provides it's own version of the llvm library, 
independant from the one of macports?

Stefan