Keep related messages in order

The short version

A message can carry a key. Kafka hashes the key to pick a partition, so messages with the same key always go to the same partition and stay in order.

Details

Messages without a key are spread across partitions. Use a key whenever order matters for one thing, such as one customer or one order. Adding partitions later changes where keys land, so choose the count with care.

Try it yourself

Send three orders, each written as key and value separated by a colon:

docker exec -it kafka /opt/kafka/bin/kafka-console-producer.sh \
  --bootstrap-server localhost:9092 --topic orders \
  --property parse.key=true --property key.separator=:
>alice:pizza
>bob:salad
>alice:cake

Read them back with their keys and partitions:

docker exec -it kafka /opt/kafka/bin/kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 --topic orders --from-beginning \
  --property print.key=true --property print.partition=true

You should see both alice messages with the same partition number, and pizza before cake.

Explain it to the duck

Why would a shop use the customer ID as the key for its order events?

Tested with: Kafka 4.1.0 in Docker.