翻译EDIT: At the suggestion of J. F. Sebastian, I can get the same error much more simply:
Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] Type "copyright", "credits" or "license" for more information. IPython 0.10 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. Welcome to pylab, a matplotlib-based Python environment. For more information, type 'help(pylab)'. In [1]: open(r'c:\test.bin', 'wb').write('a'*67076095) In [2]: open(r'c:\test.bin', 'wb').write('a'*67076096) In [3]: open(r'z:\test.bin', 'wb').write('a'*67076095) In [4]: open(r'z:\test.bin', 'wb').write('a'*67076096) --------------------------------------------------------------------------- IOError Traceback (most recent call last) C:\Documents and Settings\User\ in () IOError: [Errno 22] Invalid argument In [5]: Note that C: is a local drive, and Z: is a network drive.
ORIGINAL QUESTION:
Python 2.6.4 on Windows XP crashes if I use cPickle to write a file bigger than ~67 MB to our network drive (ReadyNAS Pro Pioneer edition). I'd like to be able to pickle large files. Is this a known problem? Is there a workaround?
The following script produces a crash:
import cPickle, numpy a = numpy.zeros(8385007) print "Writing %i bytes..."%(a.nbytes) cPickle.dump(a, open('test_a.pkl', 'wb'), protocol=2) print "Successfully written." b = numpy.zeros(8385008) print "Writing %i bytes..."%(b.nbytes) cPickle.dump(b, open('test_b.pkl', 'wb'), protocol=2) ##Crashes on a network drive print "Successfully written." ##Doesn't crash on a non-network driveHere's the steps I take to produce a crash at the ipython prompt:
Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] Type "copyright", "credits" or "license" for more information. IPython 0.10 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. Welcome to pylab, a matplotlib-based Python environment. For more information, type 'help(pylab)'. In [1]: pwd Out[1]: 'C:\\Documents and Settings\\User' In [2]: run test Writing 67080056 bytes... Successfully written. Writing 67080064 bytes... Successfully written. In [3]: cd Z: Z:\ In [4]: pwd Out[4]: 'Z:\\' In [5]: run 'C:\\Documents and Settings\\User\\test' Writing 67080056 bytes... Successfully written. Writing 67080064 bytes... --------------------------------------------------------------------------- IOError Traceback (most recent call last) C:\Documents and Settings\User\test.py in () 8 b = numpy.zeros(8385008) 9 print "Writing %i bytes..."%(b.nbytes) ---> 10 cPickle.dump(b, open('test_b.pkl', 'wb'), protocol=2) 11 print "Successfully written." 12 IOError: [Errno 22] Invalid argument WARNING: Failure executing file: In [6]: C: is the local hard drive on the machine. Z: is our network-attached storage.