JR60667: KAFKA SERVICE OCCACIONALLY FAILS TO START ON AIX ENVIRONMENT.

Download the mod pack for IBM InfoSphere Information Server 11.7.1.0 In-Place Upgrade

APAR status

• Closed as program error.

Error description

• Kafka service occacionally fails to start on AIX environment.
  
  There are the following messages in kafka-server.log when the
  error occurs.
  -----
  2019-01-29 22:28:07,967 FATAL [main] [Kafka Server 1], Fatal
  error during KafkaServer startup. Prepare to shutdown
  java.lang.RuntimeException: A broker is already registered on
  the path /brokers/ids/1. This probably indicates that you either
  have configured a brokerid that is already in use, or else you
  have shutdown this broker and restarted it faster than the
  zookeeper timeout so it appears to be re-registering.
          at
  kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:305)
          at
  kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:291)
          at
  kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:70
  )
          at
  kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:51)
          at
  kafka.server.KafkaServer.startup(KafkaServer.scala:244)
          at
  kafka.server.KafkaServerStartable.startup(KafkaServerStartable.s
  cala:37)
          at kafka.Kafka$.main(Kafka.scala:67)
          at kafka.Kafka.main(Kafka.scala)
  2019-01-29 22:28:07,972 INFO  [main] [Kafka Server 1], shutting
  down
  

Local fix

• Add 'sleep 30' before 'stopsrc -s InfoSrvZookeeper 2>&1' in
  stop-aix-services.sh.
  
  if [ ! "${COMPONENTS#*zookeeper}" = "$COMPONENTS" ]; then
      sleep 30     <<---
      stopsrc -s InfoSrvZookeeper 2>&1
      sleep 5
  fi
  

Problem summary

• Kafka is unable down properlz because yookeeper is alreadz shut
  down.
  

Problem conclusion

• Before stopping the Yookeeper service a wait statement was added
  in order to give Kafka enough time during shut down to write its
  state to Zookeeper.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WIS DATASTAGE

• Fixed component ID

  5724Q36DS

Applicable component levels

• R850 PSY

     UP