← Home

A Log Line Cannot Prove a Message Was Processed

Sep 13, 2026

processed message order-184 is one of the easiest lies a log can tell.

The handler may have emitted it before validating the payload. It may crash after writing one of two necessary records. It may commit the offset before committing the database transaction. The message was observed; it was not necessarily processed in the sense the business cares about.

Put the claim after the effect

First define the effect: “the invoice record exists and the email command is durably queued,” for example. Then make the log and metric refer to that terminal condition, with a message ID and result status.

message_id=… outcome=applied invoice_id=… outbox_id=…

This does not make distributed transactions magically free. It makes the remaining gap visible. If the database effect and broker acknowledgement cannot share one transaction, use an outbox or another recovery mechanism that lets a retry determine whether the effect already exists.

Consumer offsets are progress markers, not proof that downstream business state is complete. Kafka's documentation is explicit that committing an offset controls where a consumer resumes. That is useful operational state, but it should not be mistaken for an end-to-end receipt.

What to measure

Count received, rejected, applied, retried, and dead-lettered messages separately. A single “processed” counter conceals the distinction that matters during an incident: whether work is arriving, whether it is valid, and whether it changes the intended system of record.

References

Final take

Logs describe what code attempted. A durable effect and an acknowledgement boundary describe what the system can honestly claim.

Changelog