Python Pandas – Clean Way to Check if a Value is NaN

numpypandaspython

How can I check if a given value is NaN?

e.g. if (a == np.NaN) (doesn't work)

Please note that:

  • Numpy's isnan method throws errors with data types like string
  • Pandas docs only provide methods to drop rows containing NaNs, or ways to check if/when DataFrame contains NaNs. I'm asking about checking if a specific value is NaN.
  • Relevant Stackoverflow questions and Google search results seem to be about checking "if any value is NaN" or "which values in a DataFrame"

There must be a clean way to check if a given value is NaN?

Best Answer

You can use the inate property that NaN != NaN

so a == a will return False if a is NaN

This will work even for strings

Example:

In[52]:
s = pd.Series([1, np.NaN, '', 1.0])
s

Out[52]: 
0      1
1    NaN
2       
3      1
dtype: object


for val in s:
    print(val==val)
True
False
True
True

This can be done in a vectorised manner:

In[54]:
s==s

Out[54]: 
0     True
1    False
2     True
3     True
dtype: bool

but you can still use the method isnull on the whole series:

In[55]:
s.isnull()

Out[55]: 
0    False
1     True
2    False
3    False
dtype: bool

UPDATE

As noted by @piRSquared if you compare None==None this will return True but pd.isnull will return True so depending on whether you want to treat None as NaN you can still use == for comparison or pd.isnull if you want to treat None as NaN

Related Question