An event mechanism is a contract about delivery, ordering, retention, replay and failure. Product choice comes after that contract. Teams get into trouble when they choose Kafka, RabbitMQ or PostgreSQL first, then discover that the application needed different semantics.
I start with the lightest mechanism that meets the measured requirement. That can be server-sent events for browser updates, PostgreSQL for local signaling or change capture, a queue for work distribution, or a durable log for replay and multiple independent consumers.
Write the requirements before comparing products
- Who produces and consumes the event?
- Can a consumer be offline, and for how long?
- Must every consumer see every event, or should one worker claim each item?
- Do consumers need replay from an offset or point in time?
- What ordering scope matters: global, per key, per partition or none?
- What duplicate behavior can the application tolerate?
- What are the expected and peak event rates, payload sizes and retention periods?
- What happens when the mechanism is unavailable?
A practical mechanism comparison
| Mechanism | Strong fit | Durability and replay | Main trap |
|---|---|---|---|
| HTTP SSE | One-way browser updates | Reconnect support, but durable replay is application work | Treating a client stream as a system of record |
| PostgreSQL NOTIFY | Lightweight signaling near existing transactions | No offline consumer backlog | Sending business payloads instead of a row key |
| PostgreSQL logical decoding | Change data capture from committed database writes | Replay through replication slots and consumer offsets | Unconsumed WAL retention and OLTP coupling |
| RabbitMQ quorum queue | Work distribution, routing, acknowledgments and dead lettering | Replicated durable queue, replay is not its primary model | Ignoring publisher confirms, consumer acknowledgments or poison messages |
| RabbitMQ stream | Append-only retention and replay inside RabbitMQ | Offset-based consumption with retention | Assuming queue and stream semantics are identical |
| Apache Kafka | Durable event logs, long retention and independent consumer groups | Partition offsets and replay | Claiming end-to-end exactly-once behavior outside its transactional boundary |
| Apache Pulsar | Multi-tenant messaging with stateless brokers and BookKeeper storage | Persistent subscriptions and retained data | Underestimating the number of operating components |
| AutoMQ | Kafka-compatible workloads designed around object storage | Kafka APIs with a different storage architecture | Accepting vendor efficiency claims without a workload-specific pilot |
SSE is a client delivery channel, not a broker
The browser EventSource API opens a persistent one-way HTTP connection. It reconnects after interruption, and the event format supports an ID that a server can use to resume from a known point. The transport does not create a durable event log for you. If an offline client must recover every event, the application needs retained state and a replay endpoint behind the stream.
SSE is a good fit for dashboards, progress updates and notifications when the server is the only sender. Check proxy buffering, idle timeouts and HTTP connection limits before production. If the client also needs to send a continuous stream, choose a bidirectional protocol.
PostgreSQL offers two different event paths
LISTEN and NOTIFY for signaling
NOTIFY delivers after the transaction commits. PostgreSQL can fold duplicate notifications with the same channel and payload inside one transaction, and the default payload must be shorter than 8,000 bytes. I use it as a signal that tells a connected process which durable row to read. I do not use it as the durable record of a business event.
Logical decoding for change data capture
Logical decoding streams committed database changes through logical replication slots. It is a stronger foundation for CDC because the consumer can resume from its position. It also creates an operational obligation. A stalled slot can retain WAL and consume disk until the consumer catches up or the slot is managed. Monitor slot lag as part of database capacity, not only event-pipeline health.
RabbitMQ queues and streams solve different jobs
A quorum queue is a replicated work queue based on Raft. It fits competing consumers, acknowledgments, routing, dead lettering and poison-message handling. A RabbitMQ stream is an append-only retained log with offset-based replay. The same broker can support both, but choosing the wrong data structure changes delivery and retention behavior.
I would choose on semantics before estimating throughput. A queue should answer who owns the next unit of work and what happens after a failed delivery. A stream should answer how long records remain, how consumers address offsets and how replay affects downstream idempotency.
Kafka exactly-once claims need a boundary
Kafka provides idempotent production and transactions. Its documentation supports exactly-once processing when an application reads from Kafka, processes records and writes results back to Kafka with transactions and read-committed consumers. A write to an unrelated database, API or object store requires cooperation from that destination or application-level idempotency. The default Kafka delivery model is at least once.
That scope belongs in the architecture decision record. The phrase “Kafka gives exactly once” is too broad to test.
Pulsar and AutoMQ change the storage architecture
Pulsar uses stateless brokers for client traffic and Apache BookKeeper bookies for persistent storage. That separation supports flexible placement and multi-tenancy, but the platform still has brokers, bookies and metadata coordination to operate.
AutoMQ keeps Kafka-compatible APIs while replacing Kafka’s native log storage with its S3Stream layer. Object storage becomes the primary repository and a write-ahead log absorbs latency-sensitive writes. That can change retention economics and reassignment behavior. I would validate those benefits against the actual cloud, object store, failure model and workload rather than copying the vendor’s headline ratios into a capacity plan.
A real-time transport is a separate decision
Some control loops and device links need a transport protocol rather than a durable event platform. That is a different architecture layer. Keep the low-latency path separate from the retained business event path, then define how one feeds the other. I cover that boundary in Which Message Broker for IoT and Real-Time Control.
Pilot the failure semantics, not only throughput
- Stop a consumer long enough to create a realistic backlog, then measure catch-up time and storage growth.
- Kill a producer after send but before acknowledgment, then inspect duplicates.
- Restart a broker or database node during sustained writes.
- Inject skew so one key or partition receives a disproportionate share of traffic.
- Measure p50, p95 and p99 end-to-end latency with the required durability settings enabled.
- Test retention expiry, replay and downstream idempotency.
- Record the operator steps needed to diagnose lag and recover service.
The practical takeaway
Choose the contract first. SSE pushes updates to connected clients. PostgreSQL can signal or expose committed changes. RabbitMQ can distribute work or retain streams. Kafka and Pulsar provide durable logs for independent consumers. AutoMQ changes Kafka’s storage model. None is the default answer for every event.
If an event-platform choice will set your reliability, latency or infrastructure cost for years, I can review the requirements, failure semantics, benchmark plan and capacity assumptions through a Flash Architecture Review or Data Platform Audit. Book a 15-min intro call to discuss the decision.
FAQ
What is the first decision when choosing an event mechanism?
The delivery contract, before any product name: at-most-once, at-least-once or exactly-once within a named boundary; what durability a successful acknowledgment promises; and whether consumers need replay from history or only live tailing. Those three answers eliminate most of the field before a benchmark runs.
Is server-sent events a message broker?
No. Server-sent events is a client delivery channel: one-way HTTP push to connected browsers or clients, with no persistence model, no consumer groups and no replay history. It solves fan-out to live clients and pairs naturally with a real broker or database behind it for durability.
When does PostgreSQL beat a dedicated broker?
When the signaling volume and fan-out are modest and the data already lives in PostgreSQL. LISTEN and NOTIFY is enough for wake-up signals, and logical decoding turns committed changes into a change stream for downstream consumers. Move to a broker when consumers need independent replay, sustained high throughput, or isolation from the primary database’s workload.
Does exactly-once delivery remove the need for idempotent consumers?
No. Exactly-once claims are bounded by the system’s own transaction boundary; side effects outside that boundary (writes to another store, calls to another service) can still be repeated after a failure and retry. Idempotent consumers or deduplication keys remain necessary, and the boundary itself must be named in writing.
0 Comments