Use Case - File Forwarder Multiline

Multiline parsing is a feature of the File Forwarder plug-in for use when the file to be forwarded contains multiline blocks.

To use these additional options, you must turn on the advanced UI and set the File Parser field to Multiple lines per event.
Note: Use the Advanced UI options only under direction from IBM Support.

You can use the new fields when you intend to capture a multiline payload.

You can also include or exclude the header and footer of a block by selecting the relevant option.

You can also trim the leading and trailing white space by selecting the relevant option. The default delimiter is a space.

Table 1. Fields
Muliple lines per event    
Filter to start a multiline block field - regex - required A mask or filter finds the first line to process in multiline. You can use * (multiple characters) ? (one character) and # (a number) or a regex.
Filter to end a multiline block field - regex - required A mask or filter finds the last line to process in multiline. You can use * (multiple characters) ? (one character) and # (a number) or a regex.
Include the start and end lines of a multiline block checkbox - boolean - required You can include or exclude the first (header) and last (footer) lines of a block.
BlockTrimLines checkbox - boolean - required Trim the lines before adding to the block separated by a space.

Multiline block example

2021-11-17 09:36:07.5840|INFO|93|Running the query. Memory Limit=0, Timeout=0, RequestPriority=Normal, RequestExecutionMetrics=[None], ApplicationContext=, ConnectionCategory=ExternalOnPrem, Query=<pi>EVALUATE
  TOPN(
    101,
    CALCULATETABLE(
      VALUES('ABC'[ABC]),
      KEEPFILTERS(
        FILTER(
          KEEPFILTERS(VALUES('ABC'[Year.Key0])),
          'ABC'[Year.Key0] = DATE(2021, 1, 1)
        )
      ),
      KEEPFILTERS(
        FILTER(
          KEEPFILTERS(VALUES('ABC'[Week.Key0])),
          OR(
            OR(
              OR(
                'ABC'[Week.Key0] = DATE(2021, 10, 25),
                'ABC'[Week.Key0] = DATE(2021, 11, 1)
              ),
              'ABC'[Week.Key0] = DATE(2021, 11, 8)
            ),
            'ABC'[Week.Key0] = DATE(2021, 11, 15)
          )
        )
      )
    ),
    'ABC'[ABC],
    1
  )
 
ORDER BY
  'ABC'[ABC]</pi>| RequestID = 9025749b-f66b-7a35-3398-400d0adfb351 ClientSessionID = 99876310-dc42-c206-5e55-27a1980dcac7

Start filter example

^(\d\d\d\d[-]\d\d[-]\d\d\s\d\d[:]\d\d[:]\d\d[.]\d\d\d\d).*

This filter matches the following lines:
Lines matched by start filter example

End filter example

.*(SessionID\s=\s([\d-\w]+))

This filter matches the following lines:

Lines matched by end filter example

If you use both of these filters, the multiline parser parses everything in between them, and if selected, includes the header and footer.