Kaitai Struct Parser
The Kaitai Struct Parser processor uses a Kaitai Struct format description file to parse binary data embedded in a field and passes the parsed data to the record. You can include the Kaitai Struct Parser processor in a pipeline with an origin configured to read binary data.
Kaitai Struct is a declarative language used to describe binary data structures. A .ksy format description file describes a binary file. You can develop a format description file that describes a binary file, or you can use an existing format description file for standard binary files. For more information, see the Katai Struct website.
The Kaitai Struct Parser processor compiles a format description file, calls the compiled file to parse binary data passed in incoming records, adds fields defined in the format description file to the records, inserts the values returned by the compiled file, and passes the record to the next stage. The processor temporarily stores the compiled file while the pipeline runs. The processor deletes the compiled file when the pipeline stops.
To configure the Kaitai Struct Parser processor, you specify the Kaitai Struct format description file. You can specify the format description file by specifying the file path or by pasting the content of the file directly into the stage.
If Data Collector uses Java 8, then you must complete a prerequisite task before running pipelines with the Kaitai Struct processor.
Example
meta:
id: gif
file-extension: gif
endian: le
seq:
- id: header
type: header
- id: logical_screen
type: logical_screen
types:
header:
seq:
- id: magic
contents: 'GIF'
- id: version
size: 3
logical_screen:
seq:
- id: image_width
type: u2
- id: image_height
type: u2
- id: flags
type: u1
- id: bg_color_index
type: u1
- id: pixel_aspect_ratio
type: u1
An origin passes the binary data from a .gif file, and the processor generates the following record as output:
Prerequisite Task
If Data Collector uses Java 8, then Java Security Manager is enabled by default, and you must enable dynamic class loading from temporary folders.
Configuring the Kaitai Struct Parser Processor
Configure a Kaitai Struct Parser processor to parse binary data in a pipeline with an origin configured to read binary data.
If Data Collector uses Java 8, then you must complete a prerequisite task before running pipelines with the Kaitai Struct Parser processor.