Reprocessing rejected data files

Following a review and correction of your rejected data files, you must reprocess the messages.

About this task

Reprocessing messages can slow system performance depending upon the number of files to reprocess. Therefore, you might want to perform reprocessing when the system is least busy.

IBM® Software Support can assist you with reject data file reprocessing. Time spent might be billable on a time-and-materials basis per your support contract.

Procedure

  1. Create a new directory in your broker instance path called \Reprocess. The path should be: \data\interfaceX\Reprocess.

    You must perform this step the first time you reprocess rejected messages.

  2. Run the madconfig create_inbound_instance command. When prompted for the question “Are you reprocessing message?”, type Y (yes).
  3. Determine what your rejected files are named.

    Microsoft Windows:

    1. Open the configuration file (for example, services.ini). To locate the file, go to:

      My Computer > Properties > Advanced

    2. In the Environment Variables section, look at the MAD_CONFNAME for the path and name of the .ini file.

      Linux® or UNIX:

    3. At the command line, type echo $ MAD_CONFNAME. This setting provides the path for the configuration file.
  4. From the data\interface directory, copy the reject.toc and the reject-proc.pos files into the new Inbound instance \Inbound directory.
  5. Go to the inbound instance \Inbound directory. Rename the reject.toc file to "input.toc" and rename the reject-proc.pos file to "input-read.pos".

    Microsoft Windows:

    Right-click the file and select Rename.

    Linux or UNIX:

    Use the command: mv oldfilename newfilename

    This command directs the inbound message manager to reprocess all messages contained in the reject.dat files found in the data\interface directory. If you do not want to reprocess all rejected files, edit the input.toc in the \Reprocess directory to contain the paths and names of the files to be reprocessed. Open the original to see the convention used to include files.

  6. Set the following variables in your configuration file. Some variable requirements are dependent on the message type being processed.
    Parser dll
    Microsoft Windows: MIParserDll
    Linux or UNIX: parserDll
    Description: The parser dll is a dynamically loaded dll that determines which message type is being processed. For example, this must be set to parser_hl7 if you are processing HL7 messages.
    Connection string
    Microsoft Windows: MIConnStr
    Linux or UNIX: conn
    Description: Connection string used to connect to the operational server. Make sure to set the MAD_CTXLIB environment setting to match the connection string. The operational server configuration file is a .cfg file located in the operational server instance directory.
    Reject threshold
    Microsoft Windows: MIRejectThreshold
    Linux or UNIX: rjt
    Description: Reject message threshold. The reject threshold causes the msgbroker service to shut down when this number of consecutive reject messages has been met. The default threshold is 20.
    Input file encoding
    Microsoft Windows: MIInputFileEncoding
    Linux or UNIX: inFileEncoding
    Description: The encoding type of the messages being processed. The following encodings are valid: LATIN1, UTF8, UTF16, UTF16LE, or UTF16BE. Use this option if you are processing a file rather than a queue. By default the encoding is set to LATIN1.
    Reprocess messages
    Microsoft Windows: MIReprocessMessages
    Linux or UNIX: reproc
    Description: The reprocess messages time indicator, if set to true (1), checks the message event time against the last event time for the given member in the MDM database. If the message event time is more current than the last event time found in the database, the message is sent to operational server. This variable must be set when reprocessing rejected messages.
    DTD file directory
    Microsoft Windows: MIDTDFileDir
    Linux or UNIX: dfd
    Description: A valid directory path where the DTD is located. If the DTD file directory is not defined, the current working directory is used. This variable does not need to be defined for an HL7 Inbound Message Manager, but must defined for XML.
    Document handler dll
    Microsoft Windows: MIDocumentHandlerDll
    Linux or UNIX: docHandlerDll
    Description: The document handler optimizes the parsing of XML messages. When the document handler is not set, the Data section of the configuration file must exist with XPath definitions. By default the document handler is not set.
    Delimited token
    Microsoft Windows: MIDelimitedToken
    Linux or UNIX: delimiter
    Description: The token for the delimited messages. The default token value is ‘|'.
    Configuration file name
    Microsoft Windows: MIConfigFileName
    Linux or UNIX: cfn
    Description: A valid file name where the configuration information is defined
    Input queue directory
    Microsoft Windows: MIInputQueueDir
    Linux or UNIX: iqd
    Description: This setting is a queue directory for all input queues. This queue must point to a valid directory path that can be written to by this service.
    Input queue name
    Microsoft Windows: MIInputQueueName
    Linux or UNIX: iqn
    Description: This setting is the name used by the inbound queue and appears as a prefix for the data files in the .toc file and the *-proc.pos file.
    OK queue directory
    Microsoft Windows: MIOkQueueDir
    Linux or UNIX: oqd
    Description: This setting is a queue directory for all “good” queues. The queue must point to a valid directory path that can be written to by this service.
    OK queue name
    Microsoft Windows: MIOkQueueName
    Linux or UNIX: oqn
    Description: This setting is the name used by the "good" queue. These are messages that processed successfully.
    Reject queue directory
    Microsoft Windows: MIRejectQueueDir
    Linux or UNIX: rqd
    Description: This setting is a queue directory for all “reject” queues. The queue must point to a valid directory path that can be written to by this service.
    Reject queue name
    Microsoft Windows: MIRejectQueueName
    Linux or UNIX: rqn
    Description: This setting is the name used by the “reject” queue. These are messages that failed to process successfully.
    Default queue directory
    Microsoft Windows: MIDefaultQueueDir
    Linux or UNIX: dqd
    Description: This setting is a default queue directory for all queues. Must point to a valid directory path that can be written to by this service.
    Input file name
    Microsoft Windows: MIInputFileName
    Linux or UNIX: ifn
    Description: The input file name for the input data. Input data is read from the input file name file when the MIReadFromInputQueue is set to false.
    Read from input queue
    Microsoft Windows: MIReadFromInputQueue
    Linux or UNIX: riq
    Description: Boolean flag to determine if the input data is being read from the MIInputQueueName. By default this flag is set to true.
    Create OK queue
    Microsoft Windows: MICreateOkQueue
    Linux or UNIX: coq
    Description: Boolean flag to determine whether the OK queue is created. By default this flag is set to true.
    Filler character (fixed messages only)
    Microsoft Windows: MIFillerCharacter
    Linux or UNIX: fillerChar
    Description: The filler character for fixed messages. The default filler character is a space.
    Complete message length
    Microsoft Windows: MICompleteMessageLength
    Linux or UNIX: completeMessageLen
    Description: The complete message length of a fixed message.
    Manage duplicate attributes
    Microsoft Windows: MIManageDuplicateAttributes
    Linux or UNIX: dups
    Description: This indicator filters out matching attributes from the inbound row list. The default value is to permit duplicate attributes.
    Shutdown on complete
    Microsoft Windows: MIShutdownOnComplete
    Linux or UNIX: shutdownOnComp
    Description: The shutdown on complete indicator causes the message broker to shut down when all inbound messages have been processed.
  7. Run madconfig start_inbound_msgbroker_instance.
  8. When the message broker has processed all messages it will shut down.
  9. Verify that all messages processed successfully and that there are no reject.dat files.

Results

Reprocessing according to these steps logs any messages encountered to the MAD_LOGDIR path you specified in the updated initialization file.