Installing Lightgbm on Mac with OpenMP dependency

Question

I'm new to python and would like to install lightgbm on my macbook. I did a pip install lightgbm and it said installation successful. However when I try to import that into my notebook I get the following error message:

../anaconda/envs/python3/lib/python3.6/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
342 
343         if handle is None:
--> 344             self._handle = _dlopen(self._name, mode)
345         else:
346             self._handle = handle

OSError: dlopen(../anaconda/envs/python3/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so, 6): Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgomp.1.dylib
Referenced from: ../anaconda/envs/python3/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so
Reason: image not found

The documentation on lightgbm website gives a different installation guideline using brew install.... My question is whether I have to do a brew install? If that's the case why the pip installation shows successful installation then?

Answer 1

pip would only install lightgbm python files. The documentation states that lightgbm depends on OpenMP. So you need to install that as well. The problem you are facing is because python cannot find the required "dynamic link library" that comes with OpenMP.

brew install open-mpi and it should fix the problem.

Sidenote: As a quick test, I installed lightgbm the same way you did, and faced the same problem. But I located the libgopm.1.dylib in /usr/local/opt/gcc/lib/gcc/6. Symlinking it to the required path didn't prove to be successful.

Answer 2

For MacPorts users:

Install prerequisite ports:

port install cmake gcc7 openmpi-gcc7

Install LightGBM with pip:

export CXX=g++-mp-7 CC=gcc-mp-7
pip install lightgbm --install-option=--mpi

Check other install options like --gpu and --hdfs in the Python-package installation guide: https://github.com/Microsoft/LightGBM/tree/master/python-package

Answer 3

For Mac OS, this worked for me: https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html

macOS On macOS LightGBM can be built using CMake and Apple Clang or gcc.

Apple Clang Only Apple Clang version 8.1 or higher is supported.

Install CMake (3.12 or higher):

brew install cmake Install OpenMP:

brew install libomp Run the following commands:

git clone --recursive https://github.com/microsoft/LightGBM ; cd LightGBM
mkdir build ; cd build

For Mojave (10.14)

cmake 
-DOpenMP_C_FLAGS="-Xpreprocessor -fopenmp -I$(brew --prefix libomp)/include" 
-DOpenMP_C_LIB_NAMES="omp" 
-DOpenMP_CXX_FLAGS="-Xpreprocessor -fopenmp -I$(brew --prefix libomp)/include" 
-DOpenMP_CXX_LIB_NAMES="omp" 
-DOpenMP_omp_LIBRARY=$(brew --prefix libomp)/lib/libomp.dylib 
..

For High Sierra or earlier (<= 10.13)

cmake ..

make -j4

gcc Install CMake (3.2 or higher):

brew install cmake

Install gcc:

brew install gcc

Run the following commands:

git clone --recursive https://github.com/microsoft/LightGBM ; cd LightGBM
export CXX=g++-7 CC=gcc-7 # replace "7" with version of gcc installed on your machine
mkdir build ; cd build
cmake ..
make -j4