undefined symbol for self-built llvm opt?

Question

I write a simple llvm plugin pass that requires opt to load xxx.so file and run a ModulePass. The strange thing is that when I use deb package opt (e.g., from apt-get, let's call it opt-3.7), the plugin works fine (the drawback is that it is a Release build); however when I use the opt I build myself (simplify call it opt), it frequently complains:

Error opening 'xxx.so': xxx.so: undefined symbol: _ZNK4llvm12FunctionPass17createPrinterPassERNS_11raw_ostreamERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

using c++filt I know that opt cannot find llvm::FunctionPass::createPrinterPass(llvm::raw_ostream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const.

It's a big strange since I didn't use any FunctionPass in the pass; but let's ignore this and continue.

I then checked the result of ldd opt

linux-vdso.so.1 =>  (0x00007ffd5c1ce000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f16a90d3000)
libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f16a8ea9000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f16a8c8c000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f16a8a72000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f16a86ef000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f16a83e6000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f16a81d0000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f16a7e06000)
/lib64/ld-linux-x86-64.so.2 (0x00005645f6210000)

and ldd opt-3.7

linux-vdso.so.1 =>  (0x00007ffc51bc0000)
libLLVM-3.7.so.1 => /usr/lib/x86_64-linux-gnu/libLLVM-3.7.so.1 (0x00007fec3f725000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fec3f3a2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fec3efb1000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fec3ed97000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fec3eb79000)
libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fec3e971000)
libedit.so.2 => /usr/lib/x86_64-linux-gnu/libedit.so.2 (0x00007fec3e739000)
libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fec3e50f000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fec3e30b000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fec3e002000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fec3ddeb000)
/lib64/ld-linux-x86-64.so.2 (0x000055bad2080000)
libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fec3dbd6000)

The difference, I guess, is the so file libLLVM-3.7.so.1.

So where didn't I get wrong?

BTW, my llvm was built w/o and w/ -DLLVM_BUILD_LLVM_DYLIB=1, both has the undefined symbol issue.

Answer 1

We had exactly the same error when using LLVM plugin with CLANG 3.9. We then found the solution here: https://github.com/klee/klee/issues/336

The explanation is that ABI for libstdc++ used for LLVM and your plugin is different. To solve this problem you need to recompile your plugin with the following flag "-D_GLIBCXX_USE_CXX11_ABI=0".

Check this Makefile for LLVM plugin as example: https://github.com/SamAinsworth/reproduce-cgo2017-paper/blob/master/package/plugin-llvm-sw-prefetch-pass/Makefile