Back to explore
Data engineeringSystem flow4 min

Event streaming with Kafka

How producers, partitions, and consumers share a durable event log.

Process overview

Conceptual illustration for Event streaming with Kafka
  1. Produce
  2. Partition
  3. Store
  4. Consume
  5. React

Steps

5 steps

An event records something that happened

A producer publishes an event, such as order.created. The record contains a key, a value, and a timestamp, with optional headers.

The record enters a partition

Kafka appends events to a topic’s partitions. Ordering is guaranteed within a partition; a topic does not have one global event order.

The log retains the event

The topic stores events according to its retention configuration. Reading an event does not remove it, so another consumer can read it again.

Consumers read at their own pace

Consumers subscribe to topics and process records. Producers and consumers are decoupled, so each application can evolve around its own responsibility.

One fact can power many outcomes

The same order event can feed analytics, inventory, and notifications. Each consumer turns a shared fact into work for its own domain.

Scope

A basic Kafka event stream. Delivery guarantees, offset commits, consumer groups, retries, and transactions require additional design.

Source

Apache Kafka · Introduction(opens in a new tab)
Saved explanations