find_library() in ctypes

Question

I am trying to use the command find_library() from ctypes but I\'m getting an error that I don\'t understand its reason. I am working on Windows

This is the code:

Answer 1

Is the library you are searching for located in a common place on your machine. find_library will not perform an arbitrary search for your file system, it looks in specific places that are listed the ctypes/macholib/dyld.py module (see the dyld_find function).

If your library is in e.g. /usr/lib then it should be found, but if it is in a nonstandard location you will have to add its directory to an environment variable like DYLD_LIBRARY_PATH.

Answer 2

On Windows, find_library searches the directories in the PATH environment variable, which isn't the real search order for desktop applications that's used by the Windows loader. Notably find_library doesn't include the application directory and the current directory.

Calling Windows SearchPath would be closer, but not much closer given DLL activation contexts and other APIs such as SetDllDirectory or the newer APIs SetDefaultDllDirectories and AddDllDirectory.

Given there's no simple way to replicate the search that's used by the Windows loader, just load the DLL by name using either CDLL (cdecl) or WinDLL (stdcall):

nidaq_cdecl   = ctypes.CDLL('NIDAQmx')
nidaq_stdcall = ctypes.WinDLL('NIDAQmx')

You can add the DLL directory to PATH dynamically at runtime (in contrast to the Linux loader's caching of LD_LIBRARY_PATH at startup). For example, say your DLL dependencies are in the "dlls" subdirectory of your package. You can prepend this directory as follows:

import os

basepath = os.path.dirname(os.path.abspath(__file__))
dllspath = os.path.join(basepath, 'dlls')
os.environ['PATH'] = dllspath + os.pathsep + os.environ['PATH']

---

Alternatively, you can use a context manager that calls GetDllDirectory and SetDllDirectory to temporarily modify the search slot that's normally occupied by the current working directory. Bear in mind that, just like modifying PATH, this modifies global process data, so care should be taken when using multiple threads. An advantage to this approach is that it doesn't modify the search path that CreateProcess uses to find executables.

import os
import ctypes
from ctypes import wintypes
from contextlib import contextmanager

kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)

def check_dword(result, func, args):
    if result == 0:
        last_error = ctypes.get_last_error()
        if last_error != 0:
            raise ctypes.WinError(last_error)
    return args

def check_bool(result, func, args):
    if not result:
        last_error = ctypes.get_last_error()
        if last_error != 0:
            raise ctypes.WinError(last_error)
        else:
            raise OSError
    return args

kernel32.GetDllDirectoryW.errcheck = check_dword
kernel32.GetDllDirectoryW.argtypes = (wintypes.DWORD,  # _In_  nBufferLength
                                      wintypes.LPWSTR) # _Out_ lpBuffer

kernel32.SetDllDirectoryW.errcheck = check_bool
kernel32.SetDllDirectoryW.argtypes = (wintypes.LPCWSTR,) # _In_opt_ lpPathName

@contextmanager
def use_dll_dir(dll_dir):
    size = newsize = 0
    while newsize >= size:
        size = newsize
        prev = (ctypes.c_wchar * size)()
        newsize = kernel32.GetDllDirectoryW(size, prev)
    kernel32.SetDllDirectoryW(os.path.abspath(dll_dir))
    try:
        yield
    finally:
        kernel32.SetDllDirectoryW(prev)

For example:

if __name__ == '__main__':
    basepath = os.path.dirname(os.path.abspath(__file__))
    dllspath = os.path.join(basepath, 'dlls')
    with use_dll_dir(dllspath):
        nidaq = ctypes.CDLL('NIDAQmx')

Of course, if you're only interested in setting the DLL directory once at startup the problem is much simpler. Just call SetDllDirectoryW directly.

---

Another approach is to call LoadLibraryEx with the flag LOAD_WITH_ALTERED_SEARCH_PATH, which temporarily adds the loaded DLL directory to the search path. You need to load the DLL using an absolute path, else the behavior is undefined. For convenience we can subclass ctypes.CDLL and ctypes.WinDLL to call LoadLibraryEx instead of LoadLibrary.

import ctypes
from ctypes import wintypes

kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)

def check_bool(result, func, args):
    if not result:
        raise ctypes.WinError(ctypes.get_last_error())
    return args

kernel32.LoadLibraryExW.errcheck = check_bool
kernel32.LoadLibraryExW.restype = wintypes.HMODULE
kernel32.LoadLibraryExW.argtypes = (wintypes.LPCWSTR,
                                    wintypes.HANDLE,
                                    wintypes.DWORD)

class CDLLEx(ctypes.CDLL):
    def __init__(self, name, mode=0, handle=None, 
                 use_errno=True, use_last_error=False):
        if handle is None:
            handle = kernel32.LoadLibraryExW(name, None, mode)
        super(CDLLEx, self).__init__(name, mode, handle,
                                     use_errno, use_last_error)

class WinDLLEx(ctypes.WinDLL):
    def __init__(self, name, mode=0, handle=None, 
                 use_errno=False, use_last_error=True):
        if handle is None:
            handle = kernel32.LoadLibraryExW(name, None, mode)
        super(WinDLLEx, self).__init__(name, mode, handle,
                                       use_errno, use_last_error)

Here are all of the available LoadLibraryEx flags:

DONT_RESOLVE_DLL_REFERENCES         = 0x00000001
LOAD_LIBRARY_AS_DATAFILE            = 0x00000002
LOAD_WITH_ALTERED_SEARCH_PATH       = 0x00000008
LOAD_IGNORE_CODE_AUTHZ_LEVEL        = 0x00000010  # NT 6.1
LOAD_LIBRARY_AS_IMAGE_RESOURCE      = 0x00000020  # NT 6.0
LOAD_LIBRARY_AS_DATAFILE_EXCLUSIVE  = 0x00000040  # NT 6.0

# These cannot be combined with LOAD_WITH_ALTERED_SEARCH_PATH.
# Install update KB2533623 for NT 6.0 & 6.1.
LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR    = 0x00000100
LOAD_LIBRARY_SEARCH_APPLICATION_DIR = 0x00000200
LOAD_LIBRARY_SEARCH_USER_DIRS       = 0x00000400
LOAD_LIBRARY_SEARCH_SYSTEM32        = 0x00000800
LOAD_LIBRARY_SEARCH_DEFAULT_DIRS    = 0x00001000

For example:

if __name__ == '__main__':
    basepath = os.path.dirname(os.path.abspath(__file__))
    dllpath = os.path.join(basepath, 'dlls', 'NIDAQmx.dll')
    nidaq = CDLLEx(dllpath, LOAD_WITH_ALTERED_SEARCH_PATH)