Logstash gets Error parsing json warning messages when receiving data from Z Common Data Provider

Logstash gets Error parsing json warning messages when receiving data from Z Common Data Provider.

Symptom

Logstash gets the following warning messages when receiving data from Z Common Data Provider:

[WARN ][logstash.filters.json ][main][b033caa931fcdcdf4378522d90431e847e8a81e7042cad3ecf12e9ebe6930eea] Error parsing json {:source=>"message", :raw=> [“raw cdp data”] , :exception=>java.lang.ClassCastException: class org.jruby.RubyArray cannot be cast to class org.jruby.RubyIO (org.jruby.RubyArray and org.jruby.RubyIO are in unnamed module of loader 'app')}

Problem description

The problem occurs when the Z Common Data Provider is configured to use Generic HTTP or Generic HTTP secure protocol to send data to Logstash. When Z Common Data Provider sends data to subscribers through Generic HTTP or Generic HTTP secure protocols, the data is in ARRAY format, not JSON format.

Solution

It is recommended to use Logstash or Logstash secure protocol (TCP) to stream your data to Logstash. If Generic HTTP or Generic HTTP secure protocol must be used, remove the following JSON filter from the Logstash pipeline configuration file.

filter {
      json {
             source => "message"
     }
}