Python changed its pickle protocol to 4 in python 3.4 to 3.7 and again changed it to protocol=5 in python 3.8. How do I open older pickled files in python 3.8?
I tried:
>>> with open('data_frame_111.pkl','rb') as pfile:
... x1 = pickle.load(pfile)
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
AttributeError: Can't get attribute 'new_block' on <module
'pandas.core.internals.blocks' from '/opt/anaconda3/lib/python3.8/site-
packages/pandas/core/internals/blocks.py'>
and
>>> with open('data_frame_111.pkl','rb') as pfile:
... x1 = unpkl.load(pfile, protocol=4)
but whereas protocol is a keyword in pickle.dump it is not part of pickle.load. Instantiating pickle.Unpickler() also doesn't work. But obviously there should be a way.
In python 3.7, I would import pickle5 and use that to open newer pickles, but can't find documentation on doing the reverse in python 3.8.
with open('data_frame_111.pkl','rb') as pfile:
x1 = pickle.load(pfile)
Try changing to:
import pandas as pd
with open('data_frame_111.pkl','rb') as pfile:
x1 = pd.read_pickle(pfile)
Looks like there have been some changes due to security vulnerability concerns.
You need to upgrade to the latest version (1.3.1 worked for me) of pandas. Or, to be more precise, the pandas version when you did pickle.dump(some_path) should be the same pandas version as when you will do pickle.load(some_path).