How to add package data recursively in Python setup.py?

后端 未结 8 1883
深忆病人
深忆病人 2020-12-13 17:20

I have a new library that has to include a lot of subfolders of small datafiles, and I\'m trying to add them as package data. Imagine I have my library as so:



        
相关标签:
8条回答
  • 2020-12-13 17:40

    I can do this with a script, but seems like a super pain. How can I achieve this in setup.py?

    Here is a reusable, simple way:

    Add the following function in your setup.py, and call it as per the Usage instructions. This is essentially the generic version of the accepted answer.

    def find_package_data(specs):
        """recursively find package data as per the folders given
    
        Usage:
            # in setup.py
            setup(...
                  include_package_data=True,
                  package_data=find_package_data({
                     'package': ('resources', 'static')
                  }))
    
        Args:
            specs (dict): package => list of folder names to include files from
    
        Returns:
            dict of list of file names
        """
        return {
            package: list(''.join(n.split('/', 1)[1:]) for n in
                          flatten(glob('{}/{}/**/*'.format(package, f), recursive=True) for f in folders))
            for package, folders in specs.items()}
    
    
    0 讨论(0)
  • 2020-12-13 17:41

    If you don't have any problem with getting your setup.py code dirty use distutils.dir_util.copy_tree.
    The whole problem is how to exclude files from it.
    Heres some the code:

    import os.path
    from distutils import dir_util
    from distutils import sysconfig
    from distutils.core import setup
    
    __packagename__ = 'x' 
    setup(
        name = __packagename__,
        packages = [__packagename__],
    )
    
    destination_path = sysconfig.get_python_lib()
    package_path = os.path.join(destination_path, __packagename__)
    
    dir_util.copy_tree(__packagename__, package_path, update=1, preserve_mode=0)
    

    Some Notes:

  • This code recursively copy the source code into the destination path.
  • You can just use the same setup(...) but use copy_tree() to extend the directory you want into the path of installation.
  • The default paths of distutil installation can be found in it's API.
  • More information about copy_tree() module of distutils can be found here.

0 讨论(0)
  • 2020-12-13 17:41

    I can suggest a little code to add data_files in setup():

    data_files = []
    
    start_point = os.path.join(__pkgname__, 'static')
    for root, dirs, files in os.walk(start_point):
        root_files = [os.path.join(root, i) for i in files]
        data_files.append((root, root_files))
    
    start_point = os.path.join(__pkgname__, 'templates')
    for root, dirs, files in os.walk(start_point):
        root_files = [os.path.join(root, i) for i in files]
        data_files.append((root, root_files))
    
    setup(
        name = __pkgname__,
        description = __description__,
        version = __version__,
        long_description = README,
        ...
        data_files = data_files,
    )
    
    0 讨论(0)
  • 2020-12-13 17:44

    Use glob to select all subfolders in your setup.py

    ...
    packages=['your_package'],
    package_data={'your_package': ['data/**/*']},
    ...
    
    0 讨论(0)
  • 2020-12-13 17:52

    The problem with the glob answer is that it only does so much. I.e. it's not fully recursive. The problem with the copy_tree answer is that the files that are copied will be left behind on an uninstall.

    The proper solution is a recursive one which will let you set the package_data parameter in the setup call.

    I've written this small method to do this:

    import os
    
    def package_files(directory):
        paths = []
        for (path, directories, filenames) in os.walk(directory):
            for filename in filenames:
                paths.append(os.path.join('..', path, filename))
        return paths
    
    extra_files = package_files('path_to/extra_files_dir')
    
    setup(
        ...
        packages = ['package_name'],
        package_data={'': extra_files},
        ....
    )
    

    You'll notice that when you do a pip uninstall package_name, that you'll see your additional files being listed (as tracked with the package).

    0 讨论(0)
  • 2020-12-13 17:52

    To add all the subfolders using package_data in setup.py: add the number of * entries based on you subdirectory structure

    package_data={
      'mypackage.data.folderA': ['*','*/*','*/*/*'],
    }
    
    0 讨论(0)
  • 提交回复
    热议问题