Why write first and send later
Suppose an app records an inspection and needs to tell a supervisor’s device about it. The obvious code saves the record and then sends a message. That code has a gap. If the app crashes, the phone dies or the network drops between the two steps, the record exists but the message was never sent, and nothing will ever send it.
Chris Richardson’s description of the transactional outbox pattern names the same problem on servers: sending a message in the middle of a database transaction is unreliable because the transaction might not commit, and sending it afterwards is unreliable because the service might crash before it sends. The fix is to stop treating “save” and “send” as two separate actions.
An outbox does that. The app writes the message into durable storage as part of the same local change, so the record and the pending message either both exist or neither does. Sending becomes a separate job that can happen now, in a minute, or next week.
How the transactional outbox works
In the server version of the pattern, the outbox is a table in the service’s own database. The service inserts the message into that table in the same transaction that updates its business data. A separate process, the message relay, reads the table and publishes the messages to a broker. The pattern’s stated benefits are that messages are sent if and only if the transaction commits, and that they reach the broker in the order the application sent them.
It has one well-known side effect. The relay might publish a message and then crash before recording that it did, so after a restart it publishes the same message again. Consumers therefore need to be idempotent, for example by tracking the IDs of messages they have already processed. The article on idempotency covers how.
Outboxes on devices
Mobile and edge apps use the same idea with a local database. Android’s offline-first guidance describes “queued writes”, where the app inserts each write into a queue and drains it with exponential backoff when it gets back online, and “lazy writes”, where the app writes to local storage first and queues a network update for the earliest opportunity. On Android, draining that queue is persistent work that is often handed to WorkManager.
Older protocols work the same way. SMTP requires mail that cannot be sent immediately to be queued and retried, and each queue entry holds the envelope as well as the message. In MQTT 5.0 the client’s session state includes the messages it has sent but that have not been completely acknowledged, so they can be resent after a reconnect. NASA’s description of delay-tolerant networking uses the outbox as its analogy: each node stores data until the next node becomes available, like emails waiting in an outbox until a connection appears.
What an outbox entry holds
A useful entry contains more than the message body:
- A stable ID that stays the same across retries and restarts, so the receiver can recognise a resend.
- The destination, such as a device address, topic or endpoint.
- Delivery state: pending, in flight, delivered, failed.
- Retry bookkeeping: how many attempts have been made and when the next one is due.
- An expiry time, after which the entry is given up on and reported as failed.
The entry leaves the outbox only when there is evidence that the next party has it, normally an acknowledgment. Removing it earlier, for example as soon as the network call returns, brings back the original gap.
Bounding the outbox
An outbox is not infinite storage. A device that is offline for a long time, or that keeps sending to a recipient that never comes back, will fill it. Every outbox needs two limits: how many entries it keeps and how long each one may wait. When either limit is reached, entries fail, and the app should tell the user rather than discarding work silently.
As one concrete example, the Offline Protocol mesh SDK (v0.27.0) keeps an outbox of 500 entries with a 7-day lifetime by default, and its configuration reference notes that queue overflow and expiry produce failures rather than unlimited offline retention. While a message waits, the SDK reports message_deferred when no transport is available and message_retrying when it schedules another attempt, and only message_delivered or message_failed settles the send.
A messaging library’s outbox and an application’s outbox are not the same thing. The library’s outbox holds messages it has accepted for sending. The application still needs its own durable record of the business operation, because “delivered to the other device” is not the same as “the other application accepted and stored it”. For how a queue like this fits into syncing a whole dataset, see how offline sync works.