Decommissioning the Kafka Broker component

To remove a Kafka Broker component, first reassign the Kafka topic partition Leaders from the Kafka Broker by using the kafka-reassign-partitions.sh script, and then shutdown and delete the Kafka Broker component in Ambari.

Before you begin

Ensure that Kafka and ZooKeeper services are up and running.

Procedure

Connect to ZooKeeper Service by using the following commands:
```
cd /usr/hdp/current/kafka-broker/bin 
./zookeeper-shell.sh <zookeeper_server>:<port>
```
Where
- <zookeeper_server> is the host name where ZooKeeper is running.
- <port> is the port number where the ZooKeeper Service is running. By default, 2182.
In the ZooKeeper shell, list the brokers and their IDs by using the following command:
```
ls /brokers/ids
```

In the ZooKeeper shell, get the specific Kafka Broker information with the help of the broker IDs by using the following commands:

get /brokers/ids/<brokerId>

The output looks as follows:


"jmx_port":-1,"timestamp":"1473387779988",
"endpoints":["PLAINTEXT://<myserver.ibm.com>:6667"],"host":"<myserver.ibm.com>",
"version":2,"port":6667}
cZxid = 0xdef
ctime = Thu Sep 08 22:22:59 EDT 2016
mZxid = 0xdef
mtime = Thu Sep 08 22:22:59 EDT 2016
pZxid = 0xdef
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x1570cc10b1d0000
dataLength = 127
numChildren = 0

Exit the zookeeper shell, with the following command:
```
quit
```

Identify the list of topics and partitions that require leadership and replicas reassignment by using the following commands:

The kafka-topics.sh script is available in /usr/hdp/current/kafka-broker/bin directory.

./kafka-topics.sh --zookeeper <zookeeper_server>:2182 --describe

The output looks as follows:


Topic:__consumer_offsets        PartitionCount:50       ReplicationFactor:1     
Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
Topic: __consumer_offsets       Partition: 0    Leader: 1001    Replicas: 1001  Isr: 1001
Topic: __consumer_offsets       Partition: 1    Leader: 1001    Replicas: 1001  Isr: 1001
....
....
Topic:ambari_kafka_service_check        PartitionCount:1  ReplicationFactor:1     Configs:
Topic: ambari_kafka_service_check       Partition: 0    Leader: 1001    Replicas: 1001  Isr: 1001
Topic:events    PartitionCount:1        ReplicationFactor:2     Configs:
Topic: events   Partition: 0    				Leader: 1001    Replicas: 1001,1002     Isr: 1001,1002

Note: Topic partitions that require reassignment are identified with Leader and Replicas values that are equal to the broker ID of the node that is to be decommissioned

Isolate the topics for a specific broker ID that you want to delete by running the following commands:

For example, to decommission, broker ID 1004:

./kafka-topics.sh --zookeeper `hostname`:2182 --describe | egrep "Leader: 1004|Replicas: 1004"

You can see the following output:

	  
	  Topic: topic1    Partition: 0    Leader: 1004    Replicas: 1004    Isr: 1004
    Topic: topic2    Partition: 0    Leader: 1004    Replicas: 1004    Isr: 1004
    Topic: topic2    Partition: 2    Leader: 1004    Replicas: 1004    Isr: 1004
    Topic: topic3    Partition: 0    Leader: 1004    Replicas: 1004    Isr: 1004
    Topic: topic4    Partition: 1    Leader: 1004    Replicas: 1004    Isr: 1004
    Topic: topic4    Partition: 3    Leader: 1004    Replicas: 1004    Isr: 1004
    Topic: topic5    Partition: 1    Leader: 1004    Replicas: 1004    Isr: 1004
    Topic: topic6    Partition: 1    Leader: 1004    Replicas: 1004    Isr: 1004

Reassign partitions to a different Kafka Broker.
Stop the Kafka Broker Service and delete the component in Ambari.
Restart all the Network Performance Insight® services and HDFS and YARN that might be consuming from the decommissioned Kafka node in the cluster.