I have a data set in which there is a column known as 'Native Country' which contain around 30000 records. Some are missing represented by NaN so I thought to fill it with mode() value. I wrote something like this:
data['Native Country'].fillna(data['Native Country'].mode(), inplace=True)
However when I do a count of missing values:
for col_name in data.columns:
print ("column:",col_name,".Missing:",sum(data[col_name].isnull()))
It is still coming up with the same number of NaN values for the column Native Country.
Just call first element of series:
data['Native Country'].fillna(data['Native Country'].mode()[0], inplace=True)
or you can do the same with assisgnment:
data['Native Country'] = data['Native Country'].fillna(data['Native Country'].mode()[0])
Be careful, NaN may be the mode of your dataframe: in this case, you are replacing NaN with another NaN.
import numpy as np
import pandas as pd
print(pd.__version__)
1.2.0
df = pd.DataFrame({'Country': [np.nan, 'France', np.nan, 'Spain', 'France'], 'Purchased': [np.nan,'Yes', 'Yes', 'No', np.nan]})
| Country | Purchased | |
|---|---|---|
| 0 | NaN | NaN |
| 1 | France | Yes |
| 2 | NaN | Yes |
| 3 | Spain | No |
| 4 | France | NaN |
df.fillna(df.mode()) ## only applied on first row because df.mode() returns a dataframe with one row
| Country | Purchased | |
|---|---|---|
| 0 | France | Yes |
| 1 | France | Yes |
| 2 | NaN | Yes |
| 3 | Spain | No |
| 4 | France | NaN |
df = pd.DataFrame({'Country': [np.nan, 'France', np.nan, 'Spain', 'France'], 'Purchased': [np.nan,'Yes', 'Yes', 'No', np.nan]})
df.fillna(df.mode().iloc[0]) ## convert df to a series
| Country | Purchased | |
|---|---|---|
| 0 | France | Yes |
| 1 | France | Yes |
| 2 | France | Yes |
| 3 | Spain | No |
| 4 | France | Yes |