Message Queues & Async Processing
Decoupling a slow or unreliable step from the request path by handing it to a queue and processing it separately.
Try answering in your head first, then click a question to check the model answer.
Q1.Explain how a message queue decouples a producer and consumer, and why that matters for reliability.(show answer)
The producer publishes a message and moves on, without waiting for (or even knowing about) the consumer processing it. If the consumer is temporarily down or slow, messages simply accumulate in the queue rather than the producer's request failing or blocking — this isolates a failure or slowdown in one service from immediately cascading into the other.
Q2.Why is "exactly-once" delivery so hard to guarantee in distributed messaging systems, and what's the common practical alternative?(show answer)
Guaranteeing exactly-once requires coordinating the message delivery and the consumer's processing as a single atomic unit across a network, which is fundamentally hard with independent failures possible at any point (network partition, consumer crash mid-processing). Most real systems provide at-least-once delivery instead, and push the responsibility onto the consumer to be idempotent — safely handle the same message being processed more than once without a bad side effect (e.g. using a unique message ID to detect and skip duplicates).
Q3.What's the purpose of a dead-letter queue, and what would you do with messages that land there?(show answer)
It catches messages that failed processing after retries are exhausted, preventing them from either blocking the main queue indefinitely (a poison message that always fails could otherwise stall processing of everything behind it) or being silently lost. Messages there are typically inspected manually or by an alerting system to diagnose the root cause, then either fixed and replayed or discarded if truly invalid.
Q4.When would synchronous request/response be a better choice than an async message queue, despite the queue's benefits?(show answer)
When the caller genuinely needs an immediate result to proceed (e.g. checking if a payment succeeded before showing a confirmation) — async processing adds complexity for a benefit (decoupling, buffering load) that doesn't apply if you must wait for the outcome anyway. Async queues shine when the caller doesn't need to block on the result, or when the operation should be reliably processed even if the immediate downstream service is briefly unavailable.