Apache Kafka

Apache Kafka is a distributed event streaming platform that stores a durable, ordered log of records in partitioned topics and streams them to consumers in real time. Its defining trait is the commit log model, where producers append to a replicated partition and consumers track their own position, which supports replay, consumer groups, and exactly-once processing.

Teams use it for event pipelines, log ingestion, real-time analytics, and for decoupling services from each other.

Running Kafka on your own hardware on your own infrastructure keeps events inside your network, removes per-topic vendor pricing, and lets you replicate and back up the data streams your systems depend on.