I am learning python pandas. I see a tutorial which shows two ways to save a pandas dataframe.
pd.to_csv('sub.csv') and to open
pd.to_pickle('sub.pkl') and to open
The tutorial says
to_pickle is to save the dataframe to disk. I am confused about this. Because when I use
to_csv, I did see a csv file appears in the folder, which I assume is also save to disk right?
In general, why we want to save a dataframe using
to_pickle rather than save it to csv or txt or other format?
Pickle is a serialized way of storing a Pandas dataframe. Basically, you are writing down the exact representation of the dataframe to disk. This means the types of the columns are and the indices are the same. If you simply save a file as
csv, you are just storing it as a comma separated list. Depending on your data set, some information will be lost when you load it back up.
You can read more about pickle library in python, here.