Change Cython's naming rules for .so files

Question

I\'m using Cython to generate a shared object out of Python module. The compilation output is written to build/lib.linux-x86_64-3.5//.cpyt

Answer 1

This behavior has been defined in distutils package. distutils uses sysconfig and "EXT_SUFFIX" config variable:

# Lib\distutils\command\build_ext.py

def get_ext_filename(self, ext_name):
    r"""Convert the name of an extension (eg. "foo.bar") into the name
    of the file from which it will be loaded (eg. "foo/bar.so", or
    "foo\bar.pyd").
    """
    from distutils.sysconfig import get_config_var
    ext_path = ext_name.split('.')
    ext_suffix = get_config_var('EXT_SUFFIX')
    return os.path.join(*ext_path) + ext_suffix

Starting with Python 3.5 "EXT_SUFFIX" variable contains platform information, for example ".cp35-win_amd64".

I have written the following function:

def get_ext_filename_without_platform_suffix(filename):
    name, ext = os.path.splitext(filename)
    ext_suffix = sysconfig.get_config_var('EXT_SUFFIX')

    if ext_suffix == ext:
        return filename

    ext_suffix = ext_suffix.replace(ext, '')
    idx = name.find(ext_suffix)

    if idx == -1:
        return filename
    else:
        return name[:idx] + ext

And custom build_ext command:

from Cython.Distutils import build_ext

class BuildExtWithoutPlatformSuffix(build_ext):
    def get_ext_filename(self, ext_name):
        filename = super().get_ext_filename(ext_name)
        return get_ext_filename_without_platform_suffix(filename)

Usage:

setup(
    ...
    cmdclass={'build_ext': BuildExtWithoutPlatformSuffix},
    ...
)

Answer 2

the solution is very simple, just change one line in build_ext.py at

    def get_ext_filename(self, ext_name):

        from distutils.sysconfig import get_config_var
        ext_path = ext_name.split('.')
        ext_suffix = get_config_var('EXT_SUFFIX') 
        return os.path.join(*ext_path) + ext_suffix

change ext_suffix = get_config_var('EXT_SUFFIX') to ext_suffix = ".so" or to ".pyd" on Windows

that's it, you don't have to worry about the output name ever again

Answer 3

Seems like setuptools provides no option to change or get rid of the suffix completely. The magic happens in distutils/command/build_ext.py:

def get_ext_filename(self, ext_name):
    from distutils.sysconfig import get_config_var
    ext_path = ext_name.split('.')
    ext_suffix = get_config_var('EXT_SUFFIX')
    return os.path.join(*ext_path) + ext_suffix

Seems like I will need to add a post-build renaming action.

---

Update from 08/12/2016:

Ok, I forgot to actually post the solution. Actually, I implemented a renaming action by overloading the built-in install_lib command. Here's the logic:

from distutils.command.install_lib import install_lib as _install_lib

def batch_rename(src, dst, src_dir_fd=None, dst_dir_fd=None):
    '''Same as os.rename, but returns the renaming result.'''
    os.rename(src, dst,
              src_dir_fd=src_dir_fd,
              dst_dir_fd=dst_dir_fd)
    return dst

class _CommandInstallCythonized(_install_lib):
    def __init__(self, *args, **kwargs):
        _install_lib.__init__(self, *args, **kwargs)

    def install(self):
        # let the distutils' install_lib do the hard work
        outfiles = _install_lib.install(self)
        # batch rename the outfiles:
        # for each file, match string between
        # second last and last dot and trim it
        matcher = re.compile('\.([^.]+)\.so$')
        return [batch_rename(file, re.sub(matcher, '.so', file))
                for file in outfiles]

Now all you have to do is to overload the command in the setup function:

setup(
    ...
    cmdclass={
        'install_lib': _CommandInstallCythonized,
    },
    ...
)

Still, I'm not happy with overloading standard commands; if you find a better solution, post it and I will accept your answer.