I used cPickle and protocol version 2 to dump some computation results. The code looks like this:

> f = open('foo.pck', 'w')
> cPickle.dump(var, f, protocol=2)
> f.close()

The variable var is a tuple of length two. The type of var[0] is a list and var[1] is a numpy.ndarray.

The above code segment successfully generated a file with large size (~1.7G).

However, when I tried to load the variable from foo.pck, I got the following error.

ValueError                                Traceback (most recent call last)
/home/user_account/tmp/<ipython-input-3-fd3ecce18dcd> in <module>()
----> 1 v = cPickle.load(f)
ValueError: buffer size does not match array size

The loading codes looks like the following.

> f= open('foo.pck', 'r')
> v = cPickle.load(f)

I also tried to use pickle (instead of cPickle) to load the variable, but got a similar error msg as follows.

ValueError                                Traceback (most recent call last)
/home/user_account/tmp/<ipython-input-3-aa6586c8e4bf> in <module>()
----> 1 v = pickle.load(f)

/usr/lib64/python2.6/pickle.pyc in load(file)
   1368 
   1369 def load(file):
-> 1370     return Unpickler(file).load()
   1371 
   1372 def loads(str):

/usr/lib64/python2.6/pickle.pyc in load(self)
    856             while 1:
    857                 key = read(1)
--> 858                 dispatch[key](self)
    859         except _Stop, stopinst:
    860             return stopinst.value

/usr/lib64/python2.6/pickle.pyc in load_build(self)
   1215         setstate = getattr(inst, "__setstate__", None)
   1216         if setstate:
-> 1217             setstate(state)
   1218             return
   1219         slotstate = None

ValueError: buffer size does not match array size

I tried the same code segments to a much smaller size data and it work

Edit
Report