Split a topic into partitions

Messaging basics · Article 2 of 12 · 3 min read

Partitions are how Kafka spreads one topic across machines and readers, and every later article builds on them.

The short version

A topic is split into partitions. Each partition is an ordered log of messages stored on a broker. More partitions let more consumers read in parallel.

Details

Kafka keeps order inside a partition, not across a whole topic. You choose the number of partitions when you create a topic. You can add partitions later, but you can never remove them.

Try it yourself

docker exec kafka /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 --create --topic orders --partitions 3
docker exec kafka /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 --describe --topic orders

You should see a summary line with PartitionCount: 3, followed by one line for each partition, numbered 0, 1 and 2.

Explain it to the duck

Why does Kafka promise order inside a partition but not across a whole topic?

Tested with: Kafka 4.1.0 in Docker.