Kafka topics

Source code

Kafka provides a script kafka-topics.sh to manage topics. It simple class TopicCommand.scala. As you can see it supports create, alert, list, describe and delete commands.

Change number of partitions

/bin/kafka-topics.sh --bootstrap-server localhost:9092 --alter --topic apache_event_log_topic --partitions 4

This commands calls CreatePartitions and the corresponding backend logic starts from here. The core part is function createPartitions. There are a few constraint for this api:

1. The input topic names should not have duplicates.
2. You must have write permission on the topic.
3. The new partition number must be larger than the current number. Note, I said larger. If it is equal, it will error out.

Existing partitions will not be modified. For the new partitions, how does Kafka distribute them to brokers? This API request contract has Assignments parameter which can be used to designate assignments. If this parameter is not specified, then Kafka uses a round-robin way to do the assignment. Read StripedReplicaPlacer to learn more details.

Delete topics

/bin/kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic <topic1>,<topic2>

We can delete multiple topics in a single command. Simply separate the topics using comma. This TopicCommand class will translate , to | and then do a regex matching. See code. Basically, we can use a regex if we want to.