Using Python 2.7 and Pandas
I have to parse through my directory and plot a bunch of CSVs. If the CSV is empty, the script breaks and produces the error message:
pandas.io.common.EmptyDataError: No columns to parse from file
If I have my file paths stored in
how do I read through each one and only plot the non empty CSVs? If I have an empty dataframe defined as df= I attempt the following code
for i in range(0,len(file_paths)): if pd.read_csv(file_paths[i] == ""): print "empty" else df.append(pd.read_csv(file_paths[i],header=None))
I would just catch the appropriate exception, as a catch all is not recommended in python:
import pandas.io.common for i in range(0,len(file_paths)): try: pd.read_csv(file_paths[i]) except pandas.io.common.EmptyDataError: print file_paths[i], " is empty"
Note, as of pandas 0.22.0 (that I can be sure of) , the exception raised for empty csv is
pandas.errors.EmptyDataError. And if you're importing pandas like
import pandas as pd, then use
pd instead of
If your csv filenames are in an array
import pandas as pd for filename in manyfiles: try: df = pd.read_csv(filename) except pd.errors.EmptyDataError: print('Note: filename.csv was empty. Skipping.') continue # will skip the rest of the block and move to next file # operations on df
I'm not sure if
pandas.io.common.EmptyDataError is still valid or not. Can't find it in reference docs. And I also would advise against the catch-all
except: as you won't be able to know if it's something else causing the issue.
You can use the in built
except syntax to skip over files that return you an error, as follows:
Described here: Try/Except in Python: How do you properly ignore Exceptions?
for i in range(0,len(file_paths)): try: pd.read_csv(file_paths[i]) ### Do Some Stuff except: continue # or pass
This will attempt to read each file, and if unsuccessful continue to the next file.