I have an object that contains within it two scikit-learn models, an IsolationForest and a RandomForestClassifier, that I would like to pickle and later unpickle and use to produce predictions. Apart from the two models, the object contains a couple of StandardScalers and a couple of Python lists.
Pickling this object using joblib is unproblematic, but when I try to unpickle it later I get the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/(...)/python3.5/site-packages/joblib/numpy_pickle.py", line 578, in load
obj = _unpickle(fobj, filename, mmap_mode)
File "/home/(...)/python3.5/site-packages/joblib/numpy_pickle.py", line 508, in _unpickle
obj = unpickler.load()
File "/usr/lib/python3.5/pickle.py", line 1039, in load
dispatch[key[0]](self)
KeyError: 0
The same application both pickles and unpickles the object, so the versions of scikit-learn, joblib and other libraries are the same. I'm not sure where to start debugging, given the vague error. Any ideas or pointers?
The solution to this was pretty banal: Without being aware of it I was using the version of joblib in sklearn.externals.joblib for the pickling, but a newer version of joblib for unpickling the object. The problem was resolved when I used the newer version of joblib for both tasks.
With me, happened that I exported the model using from sklearn.externals import joblib and tried to load using import joblib.
Mine was interesting. I was working with git-lfs and thus the files had been changed and joblib couldn't open them. So I needed to run git lfs pull to get actual files. So if you are using compatible joblib versions, make sure your files are not changed somehow!