Python Windows File Copy with Wildcard Support

Question

I've been doing this all the time:

result = subprocess.call(['copy', '123*.xml', 'out_folder\\.', '/y'])
if result == 0: 
    do_something()
else:
    do_something_else()

Until today I started to look into pywin32 modules, then I saw functions like win32file.CopyFiles(), but then I found it may not support copying files to a directory. Maybe this functionality is hidden somewhere, but I haven't found it yet.

I've also tried "glob" and "shutil" combination, but "glob" is incredibly slow if there are many files.

So, how do you emulate this Windows command with Python?

copy 123*.xml out_folder\. /y

Answer 1

The following code provides a portable implementation.

Note that I'm using iglob (added in Python 2.5) which creates a generator, so it does not load the entire list of files in memory first (which is what glob does).

from glob import iglob
from shutil import copy
from os.path import join

def copy_files(src_glob, dst_folder):
    for fname in iglob(src_glob):
        copy(fname, join(dst_folder, fname))

if __name__=='__main__':
    copy_files("123*.xml", "out_folder")

Additional documentation:

• shutil.copy
• os.path.join
• glob.iglob

Answer 2

The below example is fairly naive - doesn't do any checking if something goes wrong, and doesn't create any directories, but might do what you want:

import glob
import shutil

for path in glob.iglob('123*.xml'):
    shutil.copy(path, 'out_folder/%s' % path)

See also: http://timgolden.me.uk/python/win32_how_do_i/copy-a-file.html

Using win32file/SHFileOperation copy seems to be more functional, but aren't as portable and are more complicated to use.

Answer 3

Look at glob and shutil before going any further.

Forking a subprocess to copy a file is relatively inefficient. Using shutil will be more efficient.

Answer 4

"glob" is incredibly slow if there are many files.

glob is slow is there are a lot of results because it returns one huge list of all the results in a "big bang" approach. If there are a lot of results this will use a lot of memory and take a long time.

Use iglob instead, as suggested by a previous poster. iglob constructs and returns one iterator object that can be used to loop over the results, without ever having them all in memory at the same time. It's much more efficient if there are a lot of matches.

In general, whenever you write code like "for x in [glob.glob ...]" you should be using glob.iglob instead.

Answer 5

import os
import shutil
path=os.path.join("/home","mypath")
destination=os.path.join("/destination","dir")
for r,d,f in os.walk(path):
     for files in f:
           if files.endswith(".xml"):
                 try:
                     shutil.copy(os.path.join(r,files) , destination)
                 except IOError,e:
                     print e