Working on a python scraper/spider and encountered a URL that exceeds the char limit with the titled IOError. Using httplib2 and when I attempt to retrieve the URL I receive a file name too long error. I prefer to have all of my projects within the home directory since I am using Dropbox. Anyway around this issue or should I just setup my working directory outside of home?
The fact that the filename that's too long starts with '.cache/www.example.com'
explains the problem.
httplib2
optionally caches requests that you make. You've enabled caching, and you've given it .cache
as the cache directory.
The easy solution is to put the cache directory somewhere else.
Without seeing your code, it's impossible to tell you how to fix it. But it should be trivial. The documentation for FileCache
shows that it takes a dir_name
as the first parameter.
Or, alternatively, you can pass a safe
function that lets you generate a filename from the URI, overriding the default. That would allow you to generate filenames that fit within the 144-character limit for Ubuntu encrypted fs.
Or, alternatively, you can create your own object with the same interface as FileCache
and pass that to the Http
object to use as a cache. For example, you could use tempfile
to create random filenames, and store a mapping of URLs to filenames in an anydbm
or sqlite3
database.
A final alternative is to just turn off caching, of course.
You are probably hitting limitation of the encrypted file system, which allows up to 143 chars in file name.
Here is the bug: https://bugs.launchpad.net/ecryptfs/+bug/344878
The solution for now is to use any other directory outside your encrypted home directory. To double check this:
mount | grep ecryptfs
and see if your home dir is listed. If that's the case either use some other dir above home, or create a new home directory without using encryption.
As you apparently have passed '.cache'
to the httplib.Http
constructor, you should change this to something more appropriate or disable the cache.
来源:https://stackoverflow.com/questions/14886640/ubuntu-encrypted-home-directory-errno-36-file-name-too-long