Browse
Batch vs. Streaming Processing
Processing data in scheduled chunks versus continuously as it arrives — and the latency/complexity tradeoff.
What it is
Batch processing runs on accumulated data at scheduled intervals; streaming processes each event as it arrives, continuously.
Key points
- Batch: simpler to reason about, easier to reprocess/backfill, but introduces latency equal to the batch interval — fine when "as of last night" is an acceptable freshness.
- Streaming: near-real-time results, but adds real complexity — out-of-order events, exactly-once processing guarantees, and state management across a continuously running job.
- Micro-batching (small, frequent batches) is a common middle ground that gets much of streaming's freshness with batch's simpler operational model.
- The right choice is driven by actual latency requirements, not by streaming being "more advanced" — most analytics use cases don't need sub-second freshness.