kafka常用命令

查看topic

./kafka-topics.sh --describe --zookeeper 10.200.195.9:2181 --topic test

Topic: test	PartitionCount: 2	ReplicationFactor: 1	Configs: 
	Topic: test	Partition: 0	Leader: 3   Replicas: 3	Isr: 3
	Topic: test	Partition: 1	Leader: 1   Replicas: 1	Isr: 1

增加分区

增加分区的目的是为了增加整体的吞吐量，减少单个分区的压力，增加分区的同时，需要增加consumer的数量。

./kafka-topics.sh --alter --zookeeper 10.200.195.9:2181 --topic test --partitions 3

添加结果：

WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!

• 当分区数超过broker数量：
  可以设置成功，一个broker上可以有同一个topic的多个分区。
  [root@dev-k8s-master-01 bin]# ./kafka-topics.sh –describe –zookeeper 10.200.195.9:2181 –topic test
  Topic: test PartitionCount: 5 ReplicationFactor: 1 Configs:
  Topic: test Partition: 0 Leader: 3 Replicas: 3 Isr: 3
  Topic: test Partition: 1 Leader: 1 Replicas: 1 Isr: 1
  Topic: test Partition: 2 Leader: 2 Replicas: 2 Isr: 2
  Topic: test Partition: 3 Leader: 3 Replicas: 3 Isr: 3
  Topic: test Partition: 4 Leader: 1 Replicas: 1 Isr: 1

模板版本里分区数量只支持增加，不支持减少 WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Error while executing topic command : The number of partitions for a topic can only be increased. Topic test currently has 5 partitions, 3 would not be an increase.
[2020-11-18 18:32:49,227] ERROR org.apache.kafka.common.errors.InvalidPartitionsException: The number of partitions for a topic can only be increased. Topic test currently has 5 partitions, 3 would not be an increase.

消费

• 很慢发消息的时候，消费依次均匀分配在consumer上
• 快速发的时候，一个消费者可能连续消费几条消息