ImportError: DLL load failed: The specified module could not be found — IBM DB2

问题

I know there have been many people with this same issue, but here is my situation which I have not been able to find the exact same problem. I am building an executable with pyinstaller and I keep getting the importError. I am using ibm_db package to connect to a IBM DB2 database and do inserts into a table using pandas to_sql method. I used pyinstaller on my program before I added the SQL code so I'm pretty sure it has something to do with my trying to connect to DB2, but for the life of me I cannot figure this out.

I get lots of warnings and info messages when I"m running pyinstaller but no errors that I see. I only get the error once I try to execute the executable file that pyinstaller built.

I have tried to run it in a virtual environment to try to isolate the issue but I am not that familiar with virtual environments, so I stop trying to use that.

Traceback (most recent call last):
  File "rebate_gui_sql.py", line 9, in <module>
  File "c:\users\dt24358\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
    exec(bytecode, module.__dict__)
  File "site-packages\ibm_db.py", line 10, in <module>
  File "site-packages\ibm_db.py", line 9, in __bootstrap__
  File "imp.py", line 342, in load_dynamic
ImportError: DLL load failed: The specified module could not be found.
[11020] Failed to execute script rebate_gui_sql

Update: 5/1/2019 from comment below, here is my simple program

import pandas as pd
from tkinter import *
from tkinter import ttk
import ibm_db
import ibm_db_dbi as db
from sqlalchemy import create_engine

class Application(Frame):

    def __init__(self, master):
        ttk.Frame.__init__(self, master)  
        self.master = master
        self.run_process()

    def run_process(self):
        engine = create_engine("db2+ibm_db://userid:password@url:port/database")
        conn = engine.connect()
        print("Connected to " + str(engine))

        sql = '''
            Select *
            from rebteam.pd5
            fetch first row only
            '''

        df = pd.read_sql(sql, conn)
        print(df)

        df.to_csv(r'c:\users\dt24358\scripts\pricing tool\GUI_SQL\test.csv', index=False)
        self.result_label = Label(root, text="Select of PD5 Successful", bg="light green", width=80, justify=LEFT)
        self.result_label.grid(row=0,columnspan=2)

root=Tk()
root.title("Rebate Bid Data Upload")
root.configure(background="light green")
app = Application(root)
root.mainloop()

回答1:

This answer is relevant for these versions:

pyinstaller 3.4  
setuptools 41.0.1 
ibm_db 3.0.1
ibm_db_sa 0.3.4
sqlalchemy 1.3.3

There are two separate issues here.

The immediate issue (the ImportError) is the failure to load ibm_db.dll

The ImportError happens because pyinstaller does not copy external (non python) libraries into the bundle unless you explicitly request that to happen.

The pyinstaller option --add-binary gives a workaround for this ImportError , see example below.

If your python script uses SQLAlchemy to access Db2, then you may see a second symptom at run time after building with pyinstaller. The run time symptom is either:

"sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:ibm_db_sa"

"sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:db2.ibm_db"

(depending on the prefix for the url given to create_engine())

This symptom sqlalchemy.exe.NoSuchModuleError is not specific to Db2 but can impact other databases when used via SQLAlchemy with an external dialect ( Db2, teradata, snowflake, presto,...). Databases that use SQLAlchemy internal dialects may just work out of the box.

Here is one workaround, other workarounds are possible.

SQLAlchemy external dialects use pkg_resources entry_points to allow SQLAlchemy to exploit them, but pyinstaller cannot yet handle these, without some assistance from you. Such entry point information is a kind of meta data about the module.

This workaround uses pyinstaller hooks to collect the metadata of the relevant modules , and tells pyinstaller the directory (or directories) that contain these hook files. For Db2 with SQLAlchemy, three hook files are needed, hook-ibm_db.py, hook-ibm_db_sa.py, hook-sqlalchemy.py. I choose to put these hook files in the same directory as my source file python script.

The contents of each of these files is trivial two lines, and the contents differ only by the module name contained within. Here is an example of one of the files hook-sqlalchemy.py (for the other 2 required files, just replace the module name appropriately):

from PyInstaller.utils.hooks  import  copy_metadata

datas = copy_metadata('sqlalchemy')

To add ibm_db.dll via the --add-binary method, you can either use a command line option to pyinstaller or edit the spec file.

So for both workarounds, suppose that your ibm_db.dll lives in this directory:

%LOCALAPPDATA%\programs\python\python37\lib\site-packages\ibm_db_dlls

and you make a variable to point to that location:

> set ibm_db_path=%LOCALAPPDATA%\programs\python\python37\lib\site-packages\ibm_db_dlls

For an MS-Windows batch file (using ^ as line continuation character), the pyinstaller command line example to handle both of the workarounds mentioned above is:

pyinstaller -y ^
--additional-hooks-dir=. ^
--hidden-import ibm_db_sa.ibm_db ^
--hidden-import ibm_db_dbi ^
--hidden-import ibm_db ^
--add-binary  %LOCALAPPDATA%\Programs\Python\Python37\Lib\site-packages\ibm_db_dlls\ibm_db.dll;.\ibm_db_dlls ^
 your_script.py

Notes:

If your python script explicitly imports the modules then you do not need to specify them via --hidden-import options (buy you still need the hooks).
The ibm_db.dll needs to be in subdirectory ibm_db_dlls in your bundle, which is the reason for specifying that destination on the --add-binary option.
If you are building for Linux/Unix, instead of ^. use \ as the line continuation character as usual.

回答2:

Thanks for your question and answer. I had met the same situation in Windows7 Python3.7 ibm-db 3.0.1
with your hint，I think the reason is that exe can't find *.dll in clidriver\bin and ibm_db.dll, and solve it with a similar method in two steps

Frist: the same as you, add clidriver directory to system path

**\site-packages\clidriver\bin

Second pack with argument --add-binary

 Pyinstaller --add-binary **\Lib\site-packages\ibm_db_dlls\ibm_db.dll;.\ibm_db_dlls myproject.py

Then it's OK！

similar question: PyQt5 Executable is crashing with Missing DLL

来源：https://stackoverflow.com/questions/55839589/importerror-dll-load-failed-the-specified-module-could-not-be-found-ibm-db2

标签

python-3.x

db2

pyinstaller