Unable to locate files with long names on Windows with Python

Question

I need to walk through folders with long file names in Windows.

I tried using os.listdir(), but it crashes with long pathnames, which is bad.

I

Answer 1

To locate files on UNC paths, the the magic prefix is \\?\UNC\ rather than just \\?\.

Reference: https://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx#maxpath

So to access //server/share/really/deep/path/etc/etc, you'd need to

1. Convert it to unicode (use the unicode() constructor)
2. Add the magic prefix ("\\?\\UNC\"), and
3. Ensure all directory separators are "\" (see os.path.normpath())

Resulting unicode string: \\?\UNC\server\share\really\deep\path\etc\etc

I've only experimented a little (much less than @stenci did) but with Python 2.7 it seems to work OK with os.walk(), and to fail with os.listdir().

Caveat: It only works with os.walk() if the starting path for the traversal is within the MAX_PATH limit, and none of the sub directories in the starting path would push it over the limit either. This is because as os.walk() uses os.listdir() on the top directory.

Answer 2

In my previous comment I said that the nested recursive call of GetShortPathName is not required. I found it is not required most of the times, but once in a while it crashes. I wasn't able to figure out when, so I made this little function that has been working smoothly for some time:

This is the function that I use now:

def short_name(name):
    try:
        return win32api.GetShortPathName(name)
    except win32api.error:
        dirname = os.path.dirname(name)
        basename = os.path.basename(name)
        short_dirname = win32api.GetShortPathName(dirname)
        return win32api.GetShortPathName(os.path.join(short_dirname, basename))

try:
    mtime = os.path.getmtime(name)
except FileNotFoundError:
    name = short_name(name)
    mtime = os.path.getmtime(name)

Answer 3

Use the 8.3 fallback to avoid the long pathname, browsing in Win7 explorer this seems to be what windows itself does, ie every long paths has a shorter 'true name':

>>> long_unc="\\\\K53\\Users\\Tolan\\testing\\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\\xxxxxxxxxxxxxxxxxxxxxxxxffffdffffdffffdffffdffffdffffdffffdwgggggggggggggggggggggggggggggggggggxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\\esssssssssssssssssssssggggggggggggggggggggggggggggggggggggggggggggggeee"
>>> os.listdir(long_unc)
FileNotFoundError: [WinError 3]

but you can use win32api (pywin32) to 'build' up a shorter version, ie

short_unc=win32api.GetShortPathName(win32api.GetShortPathName(win32api.GetShortPathName("\\\\K53\\Users\\Tolan\\testing\\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")+"\\xxxxxxxxxxxxxxxxxxxxxxxxffffdffffdffffdffffdffffdffffdffffdwgggggggggggggggggggggggggggggggggggxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx") + "\\esssssssssssssssssssssggggggggggggggggggggggggggggggggggggggggggggggeee")
>>> print(short_unc)
\\K53\Users\Tolan\testing\XXXXXX~1\XXXXXX~1\ESSSSS~1
>>> import os
>>> os.listdir(short_unc)
['test.txt']

clearly you can just fold the win32api.GetShortPathName call into you dir exploration rather than nesting as in my example. I've done it like this with 3 calls because if you've already got a 'too long' path then win32api.GetShortPathName wont cope with it either, but you can do it per dir and stay below the limit.