Offline-first and sync

How does offline sync work?

Offline sync lets each device work on its own copy of the data and exchange changes when a connection returns. Each device records what changed, sends the changes the other side has not seen, and applies the changes it receives, using rules that make every copy end up the same.

TL;DR

Offline sync records individual changes, uses version summaries so each replica receives only what it is missing, and applies changes so every copy converges in any order. The same mechanism works with a server, another phone over Bluetooth, or a relay. Sync only makes copies agree, so business rules still need an authority, usually the backend.

Learning objectives

After reading this article you will be able to:

  • Explain why sync systems record changes instead of sending whole state
  • Describe how version vectors show which changes each replica is missing
  • Compare a single authority, deterministic revisions and CRDTs as ways to converge

Replicas in offline sync

Each device holds its own copy of the data, called a replica, and reads and writes it directly. Sync starts from the idea that there is more than one copy of the data. When devices are connected they exchange changes quickly; when they are not, each replica moves forward on its own. Sync is the process of bringing replicas that moved apart back into line.

Two phones start with the same copy, change different fields while apart, then exchange what each did. Both end with both changes.

Change logs vs full-state sync

Most sync systems record changes because sending the whole current state is simple but heavy, and it loses information about who changed what. There are two ways to tell another device what happened. A device can send its whole current state and let the other side work out the difference. Or it can keep a log of the individual changes, often called operations or deltas, and send only those.

A change such as “set the status of job 17 to done” is small to send, and it carries enough context for the receiver to combine it with changes made elsewhere.

Version vectors and missing changes

When two replicas meet, comparing their version summaries shows which changes each has not seen. Sending everything every time would waste bandwidth, which matters on a slow or metered link.

The usual tool is a version summary. Each replica numbers the changes it makes, and keeps a small record of the highest number it has seen from every other replica, sometimes called a version vector. Comparing two summaries shows exactly which changes are missing on each side. The idea builds on Leslie Lamport’s 1978 paper on ordering events in a distributed system, which showed how to order events with logical clocks instead of trusting each device’s wall clock.

This matters because device clocks disagree. A phone that has been offline for a week may be minutes out. Sync that relies on wall-clock time alone can put changes in the wrong order.

Convergence across replicas

Sync systems apply changes so that every replica reaches the same result, whatever order the changes arrive in. Receiving the right changes is only half the job; applying them this way is the other half. This property is called convergence, and systems that guarantee it once all changes have been delivered are called eventually consistent.

There are a few common ways to get there:

  • A single authority. Devices send changes to a server, the server decides the final order, and devices take its answer. Simple, but it needs the server.
  • Revisions with a deterministic winner. Some databases, such as Apache CouchDB, keep competing revisions of a record and pick the same winner on every replica, while keeping the others available for the application to resolve.
  • Merge-friendly data types. Conflict-free replicated data types (CRDTs) are designed so that concurrent changes can be applied in any order and still produce the same result, with no coordinator.

When two changes touch the same thing, the system needs a conflict rule. How are sync conflicts resolved? covers the options.

Peer-to-peer and relay sync

The same sync mechanism works whoever the other replica is. A phone can sync with a server when it has a connection, with another phone over Bluetooth when neither has one, or with a relay that passes changes between devices that are never online at the same time. Because each exchange only sends what is missing, a change can travel through several replicas and still arrive once.

Business rules outside sync

Sync makes copies agree. It does not decide whether a change was allowed, or whether the combined result makes business sense. If two technicians each claim the last spare part while offline, both copies will converge on a state where the part is claimed twice. Rules like stock limits, exclusive assignments, and approvals still need an authority, usually the backend, and a way to tell people when an offline action was turned down.

Frequently asked questions

Why can't offline sync rely on device clocks?

Device clocks disagree, and a phone that has been offline for a week may be minutes out. Sync that relies on wall-clock time alone can put changes in the wrong order, so version summaries build on Leslie Lamport's 1978 idea of ordering events with logical clocks instead.

What is a version vector?

It is a small record each replica keeps of the highest change number it has seen from every other replica. Comparing two of them shows exactly which changes are missing on each side.

What does eventually consistent mean?

Systems that guarantee every replica reaches the same result once all changes have been delivered, whatever order they arrive in, are called eventually consistent. The property itself is called convergence.

Can two offline devices sync without a server?

Yes. A phone can sync with another phone over Bluetooth when neither has a connection, or with a relay that passes changes between devices that are never online at the same time.

Can sync stop the same item being claimed twice?

No. If two technicians each claim the last spare part while offline, both copies will converge on a state where the part is claimed twice. Rules like stock limits need an authority, usually the backend.

Sources

Build it with Offline Protocol

Offline Protocol's replicated documents sync between devices directly, over the mesh, or through a relay, and merge deterministically when copies meet. The shared state guide shows the merge rule for each kind of collection and how to test two interruptions.

Read the shared state guide