Introduction
The underlying data operations of xarray are implemented using NumPy. Missing values or NaN values are represented as np.nan in NumPy. Therefore, to handle NaN values, we must understand their characteristics and how to operate on them. In this issue, we will discuss the characteristics and handling of NaN values using xarray, as well as a data type in meteorological data that is often overlooked but is very important.
What is np.nan?
np.nan is a special floating-point value in NumPy that represents Not a Number (NaN), used to indicate missing or invalid numerical results.
import numpy as np
print(np.nan) # nan
print(type(np.nan)) # <class 'float'>Note: np.nan is of type float, not None, and not the string “nan”.
1. NaN is not equal to any value, including itself
np.nan == np.nan # False ❌
np.nan != np.nan # True ✅2. Any mathematical operation involving NaN usually results in NaN
# Assume -99.9 is a fill value in the observed data
obs_clean = obs.where(obs.tp != -99.9) # Convert to NaN
# Forecast data is usually already NaN, but for safety
pred_clean = aligned_pred.where(aligned_pred.tp.notnull())Except for
np.nan * 0 # nan (not 0!)3. How to correctly detect np.nan?
np.isnan(np.nan) # True ✅
np.isnan(1.0) # False
np.isnan(float('nan'))# True (Python native nan is also supported)
arr = np.array([1.0, np.nan, 3.0, np.nan])
np.isnan(arr) # Output: [False True False True]🔍 Common scenarios that produce np.nan
Statistical functions that ignore NaN
These functions automatically skip NaN values during calculations:
Common handling techniques
-
np.isfinite(x)
Check if the number is finite (neither NaN nor ±inf)
np.isfinite(np.nan) # False
np.isfinite(np.inf) # False
np.isfinite(3.14) # True-
np.nan_to_num(x)
Convert NaN and inf to numbers
arr = np.array([np.nan, np.inf, -np.inf, 2.0])
np.nan_to_num(arr, nan=0.0, posinf=1e6, neginf=-1e6) # Output: [0.0, 1000000.0, -1000000.0, 2.0]-
np.where() + np.isnan(): Replace NaN with related coefficients
# Replace NaN with 0
cleaned = np.where(np.isnan(arr), 0, arr)
# Or use pandas (more convenient)
# df.fillna(0)►►►
Data Type Considerations
1. Integer arrays do not support NaN
import xarray as xr
import numpy as np
# Create an integer DataArray
da = xr.DataArray([1, 2, 3], dims='x', attrs={'units': 'mm'})
# Check type
print(da.dtype) # int64
# Try to set a missing value
da[0] = np.nan
print(da.values) # [nan 2.3.] <-- Type automatically upgraded to float64!
print(da.dtype) # float64
✅ Correct approach: Convert to float first / directly use float
import xarray as xr
import numpy as np
# Create integer data
da = xr.DataArray([1, 2, 3], dims='x', attrs={'units': 'mm'})
print(f"Original type: {da.dtype}") # int32
# ✅ Correct approach: Convert to float first, then assign NaN
da = da.astype('float32') # Explicit conversion
da[0] = np.nan
print(f"Converted type: {da.dtype}") # float32
print(f"Values: {da.values}") # [nan 2. 3.]2. NaN affects all aggregation operations
np.sum([1, np.nan, 3]) # nan✅ Solution: Use np.nansum or fill/delete first
-
Pandas and xarray are NaN-friendly
Pandas uses np.nan to represent missing values by default
xarray supports NaN, and mean(dim=…, skipna=True) skips by default