Cannot convert non-finite values (na or inf) to integer

This article will explain what the Cannot convert non-finite values (na or inf) to integer’ means, what causes it, and how to resolve it. Regarding the data pipeline, it’s essential to identify and handle these values efficiently.

Let us take a look and try to understand the cause of the inability to convert non-finite values (na or inf) to integer errors and the ways to rectify them.

What generates the cannot convert non-finite values (na or inf) to integer error?

The error of not converting non-finite values (na or inf) to integers occurs when we try to convert float columns of non-finite or infinite values to integers. Several causes are responsible for converting non-finite values (na or inf) to integer errors. Take a look at them:

• Missing data that hasn’t been thoroughly cleaned
•  Examining information from sources that utilize these values as indicators of anomalies.
•  Cases where errors or exceptions arise during data processing.
•  Mathematical calculations involving division by zero or resulting in overflows.

Let us see an example using the pandas dataframe.

import pandas as pd
import numpy as np
data = {'point': [13.8, 9.7, np.nan, 2.1, np.inf]}
df = pd.DataFrame(data)
print(data)

The above example returns the output, {‘point’: [13.8, 9.7, nan, 2.1, inf]}. Let’s try converting the same values into integer values by using the following piece of code.

df['point'].astype(int)

This code snippet will result the following error.

IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer

This piece of code will raise the cannot convert non-finite values (na or inf) to integer errors. Henceforth, we cannot convert non-finite values (na or inf) to integers. The error occurred because the type integer cannot represent missing, non-finite, or finite values.

How do we resolve the inability to convert non-finite values (na or inf) to integer errors?

We can ensure that the data is properly cleaned and the data types are appropriate to avoid and address issues. To rectify this issue, safer mathematical operations are performed. It is important to correctly handle non-finite values in data to make data pipelines more robust.

Using fillna()

In Python programming, the fillna() method replaces all the null values with some stipulated values. The syntax used is dataframe.fillna(value, method, axis, inplace, limit, downcast). There are various parameters in the syntax apart from value. All other parameters are optional.

The fillna() method can be applied to a single and specific column or to the entire portion. For replacing null values and filling NaN and NA values on the entire portion, use the method Dataframe.fillna(). Here we have used fillna() on a specific column.

#dataframe.fillna(value, method, axis, inplace, limit, downcast)
df = df.fillna(value=0)
df['point'].astype('category').astype(int)

The above code snippet will return the following result.

# Returns
0                     13
1                      9
2                      0
3                      2
4   -9223372036854775808
Name: point, dtype: int64

Here notice that we have used astype() method with fillna(), this is beacuse after conversion of data type using fillna we need another method to return our converted output using astype(). Astype() is used to return the changed specified datatype.

Using errors=’ignore’ in astype()

As stated before, astype() returns the changed specified datatype. So, while using this method and specifying errors as ignore, we can solve the inability to convert non-finite values (na or inf) to integer errors.

df = pd.DataFrame({'point': [1, 2, 3, 4, 5]})
df['point'].astype('Int64',errors='ignore')

The above code snippet will return the following result.

#Returns 
0    13.8
1     9.7
2     0.0
3     2.1
4     inf
Name: point, dtype: float64

pandas.to_numeric()

As given in the name itself, this method is used to convert the given values into numeric ones. The syntax for using this method is pandas.to_numeric(arg, errors=’ignore’, downcast=None) where arg can be list, tuple, or series, errors can use ignore, raise, or other parameters, and downcast is None by default.

import numpy as np
import pandas as pd
data=pd.Series([13.8, 9.7, np.nan, 2.1, np.inf])
pd.to_numeric(data)

The above code snippet will return the following results.

# Returns
0    13.8
1     9.7
2     NaN
3     2.1
4     inf
dtype: float64

Our code has been executed successfully with the required output. Point taken here: we have used arg as a Series. You can use a series, tuple, ndarray, list, etc.

FAQs

Conclusion

We cannot convert non-finite values (na or inf) to integers because we try to convert float columns consisting of non-finite or infinite values to an integer. But substantially, we have discussed several ways, such as the fillna() method and pandas.to_numeric() method., cannot convert non-finite values (na or inf) to integer error.

References

1. fillna()
2. pandas.to_numeric()

To learn more about fixes for common mistakes when writing in Python, follow us at Python Clear.