Pickle cross platform __dict__ attribute error

问题

I'm having an issue with pickle. Things work fine between OSX and Linux, but not Windows and Linux. All pickled strings are stored in memory and sent via an SSL socket. To be 100% clear I have replaced all '\n's with ":::" and all '\r's with "===" (there were none). Scenario:

Client-Win: Small Business Server 2011 running Python 2.7
Client-Lin: Fedora Linux running Python 2.7
Server: Fedora Linux running Python 2.7

Client-Lin sends a pickled object to Server:

ccopy_reg:::_reconstructor:::p0:::(c__main__:::infoCollection:::p1:::c__builtin__:::tuple:::p2:::(VSTRINGA:::p3:::VSTRINGB:::p4:::VSTRINGC:::p5:::tp6:::tp7:::Rp8:::.

Client-Win sends a picked object to Server:

ccopy_reg:::_reconstructor:::p0:::(c__main__:::infoCollection:::p1:::c__builtin__:::tuple:::p2:::(VSTRINGA:::p3:::VSTRINGB:::p4:::VSTRINGC:::p5:::tp6:::tp7:::Rp8:::ccollections:::OrderedDict:::p9:::((lp10:::(lp11:::S'string_a':::p12:::ag3:::aa(lp13:::S'string_b':::p14:::ag4:::aa(lp15:::S'string_c':::p16:::ag5:::aatp17:::Rp18:::b.

For some reason the Windows client sends extra information along with the pickle, and when the Linux client tries to load the pickle string I get:

Unhandled exception in thread started by <function TestThread at 0x107de60>
Traceback (most recent call last):
  File "./test.py", line 212, in TestThread
    info = pickle.loads(p_string)
  File "/usr/lib64/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/lib64/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib64/python2.7/pickle.py", line 1224, in load_build
    d = inst.__dict__
AttributeError: 'infoCollection' object has no attribute '__dict__'

Any ideas?

EDIT Adding additional requested information.

The infoCollection class is defined the same way:

infoCollection = collections.namedtuple('infoCollection', 'string_a, string_b, string_c')

def runtest():
    info = infoCollection('STRINGA', 'STRINGB', 'STRINGC')
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    ssl_sock = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_TLSv1)
    ssl_sock.connect((server, serverport))
    ssl_sock.write(pickle.dumps(info))
    ssl_sock.close()

And the receiving function is much the same but does a

p_string = ssl_sock.read()
info = pickle.loads(p_string)

回答1:

Are you using different minor versions of Python? There's a bug in 2.7.3 that makes pickling namedtuples incompatible with older versions. See this:

http://ronrothman.com/public/leftbraned/python-2-7-3-bug-broke-my-namedtuple-unpickling/

回答2:

A hack, but the cross-platform issue appears to be due to namedtuples and pickle together in a cross-platform environment. I have replaced the namedtuple with my own class and all works well.

class infoClass(object):
    pass

def infoCollection(string_a, string_b, string_c):
    i = infoClass()
    i.string_a = string_a
    i.string_b = string_b
    i.string_c = string_c
    return i

回答3:

Have you tried saving as a binary pickle file?

with open('pickle.file', 'wb') as po:
    pickle.dump(obj, po)

Also - if you're porting between various OS, and if info is just a namedtuple have you looked at JSON (it's generally considered safer than pickle)?

with open('pickle.json', 'w') as po:
    json.dump(obj, po)

Edit From the ssl .read() docs it seems that .read() will only read at most 1024 bytes by default, I'll wager that your info namedtuple is going to be larger than that. It would be difficult to know how big info is a-priori I don't know if just setting nbytes=HUGE NUMBER would do the trick (I think perhaps not).

回答4:

What happens if you

p_string = ssl_sock.read(nbytes=1.0E6)
info = pickle.loads(p_string)

回答5:

Just install Pyhton 2.7.8 from https://www.python.org/ftp/python/2.7.8/python-2.7.8.amd64.msi

来源：https://stackoverflow.com/questions/14959495/pickle-cross-platform-dict-attribute-error

标签

python

cross-platform

pickle