Error occurred in XML processing error

I have never received this error before. Why am I getting these errors and how can I fix them?

Symptoms

I was saving item data (or running migrateDataToXml.sh) and got an error saying invalid XML character.

Causes

IBM® Product Master uses XML format to store item data in the database. XML has universal standards and supports only a clearly defined set of Unicode characters: Valid characters in XML
If I try to save an item with an invalid XML character in one of the attributes, then the application will not be able to store the item attribute in XML format. This can happen during:
  1. Entering data into the system for example, data entry in the user interface or import jobs
  2. Running scripts like $TOP/bin/migration/migrateDataToXml.sh. This script is run during migration to any post 10.0.0 Fix Pack 1 version of the product to create XML format of all existing data. If this script finds any invalid characters while creating XML format, it gives an error stack similar to the following:
    Error Stack for Oracle:
    java.sql.SQLException: ORA-31061: XDB error: XML event error
    ORA-19202: Error occurred in XML processing
    In line 240 of orastream:
    LPX-00217: invalid character 19 (U+0013)

            at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
            at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
            at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
            at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
            at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
            at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
            at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
            at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1044)
            at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
            at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
            at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3665)
            at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1352)
            at com.ibm.ccd.common.util.oracle.OracleUtils.mergeXML(Unknown Source)
            at com.ibm.ccd.content.common.EntryXmlProcessor.sendPendingXmlToDb(Unknown Source)
            at com.ibm.ccd.common.context.common.DBContext.ensurePendingXMLWrittenToDB(Unknown Source)
            at com.ibm.ccd.common.context.common.DBContext.commit(Unknown Source)
            at com.ibm.ccd.common.context.common.DBContext.commit(Unknown Source)

Resolving the problem

To correct this error, remove the invalid character from the item attribute. Since only a certain type of attribute can hold these characters, so we can ignore many attribute types like integers, enumerations, lookup tables, sequences and so on. These bad attributes are probably from string type attributes and can contain long strings or external URL. These invalid attributes can be introduced with a feed file import or be present in the database from previous versions.
The following list describes some of the ways to detect them:
  1. Open the item and browse through the attributes to look for the invalid character.
  2. Find the attribute through trial and error. This can be done by removing an attribute and save the item. If it gives the same error, then the attribute is good and so paste the value back. Repeat this process until you able to save the item and the last removed attribute will be the one with the bad attribute.
  3. Use the following script to print the item and browse through it to look for attributes, which might be a candidate for invalid characters:
    var ctg;
    ctg = getCtgByName("Catalog_Name");
    if(ctg != null)
    {
         out.writeln(ctg1.getEntryByPrimaryKey("pk1"));
         out.writeln(ctg1.getEntryByPrimaryKey("pk2"));
         ....
    }
  4. You can also use any third-party XML tool to detect the invalid character.
To fix this attribute, you can:
  1. Open the item in the user interface, remove the bad character from the attribute and save the item.
  2. Use SQL, WQL, or a script to delete the attribute value and set it to NULL.
  3. If there are lots of items, then write an export job to export values of all the items. Then, edit the bad attribute values, and import these values again by using a feed file or an import job.