Asset/Flow Upload to IGC Fails with 500: Internal Error

Problem

The last step of Manta Flow for IGC (upload of the lineage to IGC) occasionally fails with the following error in any of the *IgcUploadMetadataMasterScenario_*properties.log log files.

2017-04-10 16:09:26.504 [pool-2-thread-1] INFO  eu.profinit.manta.dataflow.igc.common.UploadModelTask - Inserting flows.
2017-04-10 16:12:35.358 [pool-2-thread-1] ERROR eu.profinit.manta.connector.http.caller.callback.BasicErrorCallback - Response 500: {"message":"Internal Error","code":500}.

Additional Information

Based on the information about HTTP status codes available in InfoSphere Information Governance Catalog REST API: Tips, tricks, and time-savers (the attached PDF), additional detailed information about the error is available in the Websphere log files SystemOut.log, SystemError.log, and igc.log. The location of these log files in the filesystem is described in detail in IBM Technote InfoSphere Information Governance Catalog: log files.

A typical case is low limits for objects to be processed by IGC through API. The corresponding error message is:

2017-04-10T16:12:36,084 [WebContainer : 3] INFO  com.ibm.is.igc.rest.v1.providers.APIExceptionMapper - com.ibm.is.vr.client.query.VRReachedInMemoryProcessingLimitException:
 Reached limit of processed in-memory objects.

The current request has caused 27201 objects to be loaded into memory.
Your options:
        a. Ask your WebSphere/InfoSphere system administrator to raise the JVM heap size and maxObjectsInMemory.
        b. Run your task using the Command Line Inteface (CLI).
        c. Retry your request at a later point in time, when there are less concurrent requests.

(This message is for logging/debug purposes only. The UI should compose its own, translatable text.)

Solution

Investigate the error message in the above mentioned log files. If the message says "Reached limit of processed in-memory objects", the solution is:

Otherwise, check for solutions in the IBM Knowledge Center or with IBM Support.