Alex Rivera | Logout

How to replace NaN values in a dataframe column

Asked 2012-11-08T18:50:39.703
679

I have a Pandas Dataframe as below:

      itm Date                  Amount 
67    420 2012-09-30 00:00:00   65211
68    421 2012-09-09 00:00:00   29424
69    421 2012-09-16 00:00:00   29877
70    421 2012-09-23 00:00:00   30990
71    421 2012-09-30 00:00:00   61303
72    485 2012-09-09 00:00:00   71781
73    485 2012-09-16 00:00:00     NaN
74    485 2012-09-23 00:00:00   11072
75    485 2012-09-30 00:00:00  113702
76    489 2012-09-09 00:00:00   64731
77    489 2012-09-16 00:00:00     NaN

When I try to apply a function to the Amount column, I get the following error:

ValueError: cannot convert float NaN to integer

I have tried applying a function using math.isnan, pandas' .replace method, .sparse data attribute from pandas 0.9, if NaN == NaN statement in a function; I have also looked at this Q/A; none of them works.

How do I do it?

python pandas dataframe nan fillna
Edit
Report

1 Answer

1020

DataFrame.fillna() or Series.fillna() will do this for you.

Example:

In [7]: df
Out[7]: 
          0         1
0       NaN       NaN
1 -0.494375  0.570994
2       NaN       NaN
3  1.876360 -0.229738
4       NaN       NaN

In [8]: df.fillna(0)
Out[8]: 
          0         1
0  0.000000  0.000000
1 -0.494375  0.570994
2  0.000000  0.000000
3  1.876360 -0.229738
4  0.000000  0.000000

To fill the NaNs in only one column, select just that column.

In [12]: df[1] = df[1].fillna(0)

In [13]: df
Out[13]: 
          0         1
0       NaN  0.000000
1 -0.494375  0.570994
2       NaN  0.000000
3  1.876360 -0.229738
4       NaN  0.000000

Or you can use the built in column-specific functionality:

df = df.fillna({1: 0})
answered 2012-11-08T18:54:27.467

Your Answer