Kestrel is a simple, distributed message queue system built originally by Twitter. Its design is relatively lightweight and is engineered for speed and simplicity. Kestrel supports queuing patterns such as enqueue, dequeue, and delayed re-enqueue (for example, when a consumer fails to process a message). It stores messages persistently on disk with a memory-backed cache, allowing recovery in case of failures. Because it is intended for relatively simple use cases, it does not provide the full feature set of some enterprise messaging systems, but is often sufficient for many asynchronous or buffered workloads. Over time, the project became inactive and is now archived. Its minimalism and ease of integration made it appealing for smaller or more controlled message-queueing needs.
Features
- Implements durable (journaled) queues so that messages are preserved even in case of failures
- Supports multiple protocols (memcache protocol and thrift protocol) for compatibility and flexibility
- Item expiration (messages can be automatically expired after time)
- Fan-out queues: one writer, many readers, to broadcast messages to several consumers
- Ordered queue semantics (messages delivered in insertion or defined ordering)
- Simple, lightweight, designed for performance and reliability in message queue use-cases
Project Samples
