Replication
replicationKafka replicates each partition across multiple brokers for fault tolerance. One broker is the leader (handles all reads and writes). The rest are followers (replicate from the leader). The ISR (In-Sync Replicas) is the subset of followers fully caught up. min.insync.replicas controls how many ISR members must acknowledge a write before it is considered successful.
Details
Kafka's replication model provides both durability and high availability.
Leader election: One broker per partition is elected leader by the KRaft controller (or ZooKeeper in legacy mode). All produce and fetch requests go to the leader. If the leader fails, one of the ISR followers is elected as the new leader.
ISR (In-Sync Replicas): A follower is 'in-sync' if it has replicated all messages within replica.lag.time.max.ms (default 30 seconds). Followers that fall behind are removed from ISR. Once caught up, they rejoin ISR. A partition with fewer ISR members than min.insync.replicas will reject produce requests with acks=-1 (NOT_ENOUGH_REPLICAS error).
Replication factor and durability: replication.factor=1: no redundancy. Broker failure = data loss. replication.factor=2: survives 1 broker failure. Not recommended (split-brain risk). replication.factor=3: survives 1 broker failure (2 ISR). Production standard. Recommended: replication.factor=3, min.insync.replicas=2, acks=-1 (all ISR must acknowledge)
Unclean leader election: unclean.leader.election.enable=false (default): only ISR members can become leader. Safer – no data loss. unclean.leader.election.enable=true: any replica can become leader. Faster recovery but risks data loss if the new leader is behind.
Preferred replica: Kafka elects the first replica in the partition replica list as the preferred leader. If this broker is available, leadership shifts back to it (auto.leader.rebalance.enable=true). This keeps the leader distribution even across brokers.
Key facts
- →
All reads and writes go to the partition leader – followers never serve consumer requests directly
- →
ISR shrinks when followers fall behind; a write is durable only when acknowledged by all ISR members (acks=-1)
- →
min.insync.replicas=2 with replication.factor=3 is the standard production durability configuration
- →
KRaft (Kafka 3.3+) replaces ZooKeeper for controller election and metadata management
Common gotchas
replication.factor=2 is not safe – if one replica is the leader and falls behind before failing, unclean leader election can promote the lagged follower, losing messages.
min.insync.replicas=1 with acks=-1 provides no stronger durability than acks=1 – it only requires the leader to acknowledge.
Partition leader imbalance after broker restarts causes uneven load. Enable auto.leader.rebalance.enable=true or manually trigger preferred replica election.