Known limitations and workarounds in InfoSphere Information Analyzer Version 11.7

Content

The following are known limitations in the 11.7 release. For information about the limitations in the 11.5 which also apply to the 11.7 release, see http://www.ibm.com/support/docview.wss?uid=swg21996131.

When a user selects the Discover option for a data connection and provides one or more incorrect or invalid names in root assets, the metadata import fails and the discover process fails internally. However the message in product says that the process was successful.

Workaround: There is no work around for this issue.

When a column analysis is run in the single frequency distribution table (SingleFD) mode with the CAPTURENONE option, the old results are not cleaned up and will still show up, if Column analysis was previously run for those columns with the CAPTUREALL or CAPTUREN options.

Workaround: There is no work around for this issue.

When viewing the results for the columns for which a column analysis is run in single frequency distribution table (singleFD) mode, the Rebuild Inferences and Reference Tables buttons, in the Open Column Analysis -> View Details menu are disabled.

Workaround: There is no work around for this issue.

Running a data rule takes long time when the autocommit mode is set to true.

Work around: Set the autocommit mode as false, This can either be done during runtime for a specific run or using the InfoSphere Information Analyzer HTTP API or command line interface at the project or workspace level. You can change the autocommit setting at the global level by updating the DataStage engine (dsengine) settings.
Note: Column analysis performance is better when autocommit mode is true.

A blank page is displayed when users with limited user roles try to access a data connection in the thin client instead of generating an error message that says that a user does not have the proper roles to access the page.

Workaround: There is no work around for this issue.

The IAAdmin command "-setCredentialMapping" sets an encrypted password when it is run for the first time for a data connection. Jobs fail to run with the new credentials.

Workaround: Run the same command for the second time. This will set the right credentials and the jobs should run successfully.

A data connection created using the 'CustomURL' option in the thin client generates an error message after the connection is created and you browse for the connection. When you click on the Connections tab, none of the data connections appear.

Workaround: The following two steps will allow you to see the data connections on the Connections tab:

• Log into InfoSphere Governance Catalogue (IGC) and delete the host with the <No Name> entry.
• Log into InfoSphere Metadata Asset Manager (IMAM) and delete the host with empty name under this specific import area.

To use an HDFS connection with a custom URL to import data sets or to add datasets to a workspace, use InfoSphere Metadata Asset Manager to first import your datasets to the Catalog through this connection. Then use the IA thin client to find the files in the Catalog and add the selected files to a workspace.

Errors are generated during installation related to the report templates. The following error is generated in the installation log:
2017-12-03T05:55:01.550, INFO: CDIIN4482I: End: [actionId = init.add.roles, installUnitId = IAServices]
2017-12-03T05:55:01.550, INFO: CDIIN4480I:Begin: [actionId = ia-report-template-analysis-publication, installUnitId = IAServices]
2017-12-03T05:55:01.552, INFO: CDIIN2753I:Command Array: /bin/sh /opt/IBM/InformationServer/ASBServer/bin/ReportingAdmin.sh -url https://<server>:<port> -authfile /tmp/isadmin.authfile -deploy -templates -file /opt/Builds/11700/is-suite/payloads/InformationAnalyzer/IAServices/resources/templates/reports/ia-report-template-analysis-publication.jar, env = null, workingDir =/opt/IBM/InformationServer/ASBServer/bin
2017-12-03T05:55:07.341, INFO: Exit Code: 0, Error Encountered the following service exception: [com.ibm.iis.xmeta.client.exception.ServiceException: No active transaction detected]
2017-12-03T05:55:07.341, INFO: getLanguage for language code en and country code
2017-12-03T05:55:07.342, INFO: getLanguage for language code en and country code
2017-12-03T05:55:07.344, SEVERE: com.ibm.is.install.engine.runtime.InstallEngineˆM
com.ibm.is.install.exception.ActionException: CDIIN5125E: ExecuteScriptAction failed: the error output matched the standard error regular expression RegularExpression (
_pattern = SubstitutableValue: Token=".+", Value=".+"
).:
Error Encountered the following service exception: [com.ibm.iis.xmeta.client.exception.ServiceException: No active transaction detected]

CDIIN5106I: Details:
InstallAction (_id = ia-report-template-analysis-publication, _installUnitModelId = IAServices)
at com.ibm.is.install.engine.action.builtin.AbstractExecuteAction.execute(AbstractExecuteAction.java:343)
at com.ibm.is.install.engine.runtime.InstallEngine.executeAction(InstallEngine.java:1598)
at com.ibm.is.install.engine.runtime.InstallEngine.processInstallActionStep(InstallEngine.java:1538)
at com.ibm.is.install.engine.runtime.InstallEngine.processActionStep(InstallEngine.java:1422)
at com.ibm.is.install.engine.runtime.InstallEngine.processInstallStep(InstallEngine.java:587)
at com.ibm.is.install.engine.runtime.InstallEngine.processAllSteps(InstallEngine.java:567)

Workaround: Deploy the problematic report template manually using the following command:

<ISHOME>/ASBServer/bin/ReportingAdmin.sh -url https://<server>:<port> -authfile /tmp/isadmin.authfile -deploy -templates -file <is-suite-location>/payloads/InformationAnalyzer/IAServices/resources/templates/reports/ia-re port-template-analysis-publication.jar

After you upgrade InfoSphere Information Server from a previous version such as 11.5, 11.5.0.1 or 11.5.0.2, 11.7 and subsequent versions, you might see an error dialog box with the following information in the thin client while browsing the "Workspace settings" option in the workspaces or projects created in previous versions:

Error
The action could not be completed because the following error occurred on the server: Contact your administrator.

com.ibm.iis.dq.da.exception.DAException

Workaround: You can ignore this error message.

When a HDFS file is being analyzed by the Spark engine, if an escape character is not specified, then the default escape character ("\"), is considered.
In instances where the default escape character is considered, records such as "a","b",10,"0.5","test\","test2" can be misinterpreted. For example, if you have the 6 columns a,b,10, 0.5, test\, test2, the Spark engine interprets the columns as 5 columns (a,b,10,0.5,"test",test2").

Workaround: If you have values that end with backslash ("\"), make sure you escape them by inserting another backslash. For example, your columns should be "a","b",10,"0.5","test\\","test2".

An analysis on HDFS files or Hive tables might fail because the APT_IOPort parameter read failed during the InfoSphere Information Server on Hadoop setup.

Symptoms: The following entries are logged in the ASBNode/logs/odfenginelog.out file:

Event 36: pxbridge(1),1: Virtual data set.; input to "pxbridge(1)": getRecord DM read error; returning false.
Event 37: pxbridge(1),1: Operator terminated abnormally: runLocally() did not return APT_StatusOk
Event 38: pxbridge(1),1: JVMDUMP039I Processing dump event "abort", detail "" at 2018/04/18 11:55:52 - please wait.

Work around: Set the value for the APT_RECVBUFSIZE environment variable in the DataStage Administrator to 65536. Then, re-run the analysis.

Analysis jobs for some of the HDFS files that use InfoSphere Information Server on Hadoop setup fails with the fatal error, "Could not read the host:port."

Symptoms: The DataStage Director log contains the following entries:
Entry 30: main_program: Fatal Error: Could not read the host:port details of the Application Master from PX YARN Client: Error getting the ApplicationMaster details. Look into YARN Client's logs for more information at /opt/IBM/InformationServer/Server/PXEngine/logs/yarn_logs/yarn_client.dsadm.out*

Workaround: The following InfoSphere Information Server on Hadoop resources need to be tuned:

• Number of available cores
• RAM availability
• Size of the data involved in analysis jobs

To control the number of parallel jobs:

• Increase the value for the com.ibm.iis.ia.server.jobs.columnsPerTable.max property to 50 or more to reduce the number of DataStage jobs generated per table/file.
  For example:
  <ISHOME>/ASBServer/bin/iisAdmin.sh -set -key com.ibm.iis.ia.server.jobs.columnsPerTable.max -value 50

• Control the maximum running job count in the Operations Console

Analysis jobs for Oracle sources containing CLOB columns fails with the fatal error when they are imported using JDBC Connector. This issue occurs in 11.7 / 11.7.0.1.

Symptoms: The DataStage Director log contains the following entries:
pxbridge(0),0: Fatal Error: The connector detected character data truncation for the link column POPULATION_DESCRIPTION. The length of the value is 690 and the length of the column is 200.

Work around: These sources can be analyzed after importing them using Oracle or ODBC Connector.

Information Analyzer analysis job fails with resource constraints.

Symptoms: The InfoSphere DataStage Director log contains the following entries:
"message": "DataStage job failed. Check the log for more details.\nDataStage job log:\nEntry 123: node_node1: Fatal Error: Unable to start ORCHESTRATE process on node node1 (respin2-node-2.fyre.ibm.com): APT_PMPlayer::APT_PMPlayer: fork() failed, Resource temporarily unavailable"

Fix: Increase the soft nproc value for a non-root user to 4096 in the file 90-nproc.conf or 20-nproc.conf, which is in /etc/security/limits.d/ folder.

This value should be increased to a higher value, like 10240, when the number of concurrent jobs to be executed are more..

Information Analyzer Analysis jobs fail with JVM related errors.

Symptoms: The DataStage Director log contains the following entries:
"Entry 41: pxbridge(6),0: The JVM could not be created.  Error code:-4 (::createJVM, file CC_JNICommon.cpp, line 652)
Entry 52: node_node1: Player 8 terminated unexpectedly.
Entry 53: main_program: APT_PMsectionLeader(1, node1), player 8 - Unexpected termination by Unix signal 6(SIGABRT).
Entry 54: main_program: APT_PMsectionLeader(2, node2), player 6 - Unexpected exit status 1."

Fix: Use the iisAdmin command to set the com.ibm.iis.ia.engine.javaStage.heapSize property to a value, such as 512MB. The default value is 1024.

Example: <ISHOME>/ASBServer/bin/iisAdmin.sh -set -key com.ibm.iis.ia.engine.javaStage.heapSize -value 512

P.S.: This value depends on the amount of data to be processed in the requests. This value needs to be increased when the amount of data to be processed is greater.

Information Analyzer Analysis Data Quality analysis job fails when the number columns in the source tables/files is more than 1012 and Information Analyzer analysis database is configured as Db2.

Symptoms: The DataStage Director log contains the following entries:
"Entry 27: pxbridge(7): [Input link 0] SQLExecDirect() reported: SQLSTATE = 54011: Native Error Code = -680: Msg = [IBM][CLI Driver][DB2/LINUXX8664] SQL0680N The statement failed because too many columns were specified for a table, view, table function, or nickname, or because too many fields were specified for a user-defined row data type. SQLSTATE=54011
Entry 29: main_program: Fatal Error: [Input link 0] An error occurred when creating the table. Statement = create table DB2CCTEMP_20180531135836021439000000."

Workaround: Use the iisAdmin command to set the com.ibm.iis.ia.server.jobs.loadQualityProblemResults property to false. This setting avoids persisting the details of the Data Quality problem rows to the Information Analyzer database, which will not allow the user to drill down into the details of the data quality results in the thin client. However, you are able to see most of the analysis results.

Discovery never completes for some datasets when 'Publish' task is included with other tasks

Workaround: Exclude publish task and complete rest of the tasks. Use the publish task alone in the next run.

Purge Frequency Distribution results feature does not take the input as a fully qualified column name which has host name with "." in it. This is an issue for columns from tables only, and not for columns from files.

Workaround: There is no work around.

Analysis jobs for Oracle sources containing CLOB columns fails with the fatal error when they are imported using Oracle Connector. This issue occurs in 11.7.0.1 Service Pack 1.

Symptoms: The DataStage Director log contains the following entries:
" Entry 283: pxbridge(0),0: The OCI function OraOCILobRead2 returned status -1. Error code: 24806, Error message: ORA-24806: LOB form mismatch. (CC_OraBigBufferHandler::callbackGetDataFromOCILobLocator, file CC_OraBigBufferHandler.cpp, line 5,943) Entry 288: pxbridge(0),0: The runLocally() of the operator failed. Entry 290: pxbridge(0),0: Operator terminated abnormally: runLocally() did not return APT_StatusOk Entry 319: node_node1: Player 1 terminated unexpectedly. Entry 320: main_program: APT_PMsectionLeader(1, node1), player 1 - Unexpected termination by Unix signal 9(SIGKILL). Entry 321: sample,0: Failure during execution of operator logic. Entry 324: APT_CombinedOperatorController(0),0: Fatal Error: waitForWriteSignal(): Premature EOF on node ug-dn-16.svl.ibm.com Bad file descriptor Entry 325: main_program: APT_PMsectionLeader(1, node1), player 6 - Unexpected exit status 1. APT_PMsectionLeader(1, node1), player 7 - Unexpected exit status 1. APT_PMsectionLeader(1, node1), player 8 - Unexpected exit status 1. APT_PMsectionLeader(1, node1), player 9 - Unexpected exit status 1. APT_PMsectionLeader(1, node1), player 10 - Unexpected exit status 1. "

Workaround: These sources can be analyzed after importing them using JDBC or ODBC Connector.

In 11.7 Fix Pack 1, and later, you must have IA Data Admin role to add data sets to a workspace. This role is required to run automated discovery.

When an asset which is impacted by an automation rule is deleted, the status of the rule doesn't change to ERROR automatically. The status is updated to ERROR only when you edit the automation rules. The following asset types can be deleted: Label, Host, Database, Workspace, Attribute, Data Class, Data Rule Definition.

When you import a table by using the Snowflake connector and run analysis on this table, the analysis fails with the error "The connector failed to set the specified isolation level: TRANSACTION_READ_UNCOMMITTED".

Workaround: Use ODBC connector to connect to the Snowflake database, and run analysis again.

New Information Governance Catalog UI allows Information Analyzer users with roles IA Data Administrator, IA Project Administrator, and IA User to log in and display the home page. However, when such users try to access assets, for example by navigating to Catalog and searching for selected types, the error 'Cannot read property 'map' is displayed. It means that these users cannot access these pages.

Solution: Users should also have one of Information Governance Catalog roles.

You're unable to access formats for columns with type DATE_TIMESTAMP in file data. When you open Workspaces > specific workspace > View All > specific data set > column > Formats menu > specific format, an error 'Parameter value for FORMAT not supported' occurs.

Workaround: There is no workaround for this issue.

Quality score that is generated as a result of Data Quality analysis on data sets in one of the default workspaces, for example UGDefaultWorkspace, remains 100%, no matter how bad the data is. The issue occurs because the settings for data quality dimensions are set incorrectly. As a result, all the dimensions are disabled in Workspaces > UGDefaultWorkspace > Workspace settings > Data quality.  The issue occurs only when you upgrade from Information Server 11.7.0.2 to 11.7.0.2 SP1.

Workaround: To resolve the issue, download the attached file, extract it, and run the following command from <ISHOME>/ASBNode/bin or <ISHOME>/ASBServer/bin.

useAutomaticConfiguration.zip

./IAAdmin.sh -setDataQualityConfig -projectName UGDefaultWorkspace -content useAutomaticConfiguration.json -user <isuser> -password <ispassword>

Publishing of analysis results of large number of columns (for example more than 1000) fails with a gateway error. The issue occurs when IA thin client is launched from the Enterprise Search node launchpad, for example https://<ESNodeHostname>/ibm/iis/dq/da/index.html. The publish option for multiple data sets is available in IA thin client in Workspaces > specific workspace > View All > datasets.

Workaround: Publish the analysis results from the IA thin client launched from Information Server node launchpad, for example https://<ISNodeHostname>:9446/ibm/iis/dq/da/index.html.

When you try to access data sets in Information Analyzer thin client or in Information Governance Catalog New, an SQL exception error occurs. This happens only when SQL Server is used as metadata repository.

There is no workaround.