You are using a native SQLI connector to access an Informix database. You have used the "insert the code" feature in your Jupyter notebook to retrieved and show the data, but running the notebook cell produces a NumberFormatError:
Caused by: java.sql.SQLException: System or internal error java.lang.NumberFormatException: For input string: "customer_num".
Diagnosing The Problem
This seems more like an issue in the data itself which is not able to convert. The Spark JDBC reader inspects a subset of the data (not the metadata) and determines the data type based on that. So the error might be in the data itself.
The stack trace suggests that the Spark JDBC reader tries to read a long value (IfxResultSet.getLong), but hits a string value, so a NumberFormatException is thrown(NumberFormatException: For input string: "customer_num").
Resolving The Problem
The solution is to add a connection property to the URL (DELIMIDENT=Y).
You can add
.option('DELIMIDENT','Y') to the read method. Consider the syntax with regards to indentation and continuation (
11 January 2021