Encryption and identity

MLS vs the Signal Protocol

The Signal Protocol's published specifications set up and run encrypted sessions between two parties. X3DH or PQXDH agrees a key, even with an offline recipient, and the Double Ratchet derives a new key for every message. MLS (RFC 9420) is a group protocol from the start. Members share one evolving group secret, and a ratchet tree keeps the cost of changing it proportional to the logarithm of the group size, at the price of every member agreeing on the order of group changes.

TL;DR

Signal's specifications protect two-party sessions while MLS is built around a group, so MLS makes group key updates and compromise recovery cheaper than sender keys. In exchange, MLS members must all apply the same Commit each epoch and MLS makes no deniability claims, while Signal has added post-quantum key agreement. The choice depends on group size, ordering and deniability.

Learning objectives

After reading this article you will be able to:

  • Compare how X3DH and MLS key packages start a conversation with an offline recipient
  • Explain why sender keys scale poorly and how the MLS ratchet tree differs
  • Distinguish MLS and the Signal Protocol on ordering, deniability and post-quantum work

MLS vs Signal Protocol design

MLS and the Signal Protocol both aim at end-to-end encryption that works when people are not online at the same time, with forward secrecy and recovery after a compromise. The difference is the unit they are built around: a pair of people, or a group.

The Signal Protocol is a family of specifications published by Signal at signal.org/docs. They include X3DH and its post-quantum extension PQXDH for agreeing a key, the Double Ratchet for protecting the messages that follow, Sesame for managing sessions across a user’s devices, and XEdDSA for signatures. The key agreement and ratchet specifications describe communication between two parties; none of them defines a group key agreement.

MLS, the Messaging Layer Security protocol, is an IETF standard published as RFC 9420 in July 2023, with its architecture in RFC 9750. It defines key agreement for a group, which can be as small as two members or as large as thousands.

Prekeys and KeyPackages for offline recipients

MLS and the Signal Protocol both use keys published in advance so a sender can start without waiting for an offline recipient.

In X3DH, the recipient uploads an identity key, a signed prekey and a set of one-time prekeys to a server. The sender fetches a prekey bundle, performs several Diffie-Hellman calculations against those keys, and derives a shared secret it can use to encrypt a first message straight away. PQXDH extends this with a post-quantum key encapsulation mechanism, giving post-quantum forward secrecy, while mutual authentication still relies on the hardness of the discrete log problem.

In MLS, each client publishes KeyPackages through the Delivery Service. A member who wants to add someone uses their KeyPackage to add them to the group and sends them a Welcome message with the group’s current state. How does MLS work? goes through this in detail.

Key refresh in the Double Ratchet and MLS

The Double Ratchet derives a new key for every message, so earlier keys cannot be calculated from later ones. Each message header also carries the sender’s current ratchet public key, and the parties take turns replacing those keys and mixing the new Diffie-Hellman outputs into their chains. That lets a session recover after an attacker briefly learns a party’s keys, which the specification calls break-in recovery.

MLS refreshes keys per epoch rather than per exchange. A member gains forward secrecy within an epoch by deleting each message key after use, and across epochs by deleting old tree keys. Recovery after a compromise comes when a member updates its leaf key in a Commit. RFC 9420 suggests doing this on the order of hours or days, not with every message.

Group messaging in Signal vs MLS

With only two-party sessions, as in the Signal specifications, a group has two options, both described in the introduction to RFC 9420: pairwise channels or sender keys. Members can encrypt each message separately for every other member over pairwise channels. Or each member can distribute a symmetric sender key over those channels and encrypt group messages once with it. Sender keys are efficient and give forward secrecy with a hash ratchet, but RFC 9420 says recovering from a compromise with them is difficult: it needs a number of key update messages that grows with the square of the group size, and an attacker who learns a sender key can often keep reading that member’s messages.

MLS places members in a ratchet tree instead, which lets the group derive and update its shared keys at a cost that scales with the logarithm of the group size. The RFC 9420 introduction gives this scaling problem as the motivation for the design.

Signal specificationsMLS (RFC 9420)
Unit of protectionA session between two partiesA group of two or more members
Key refreshA new key per message, and a Diffie-Hellman step when the other side repliesEach epoch, started by a Commit
Group key updatesNot defined in the published specificationsRatchet tree, logarithmic in group size
Ordering neededOut-of-order messages handled with skipped keysMembers must agree on one Commit per epoch
DeniabilityStated goal of X3DH and PQXDHNo claims made

MLS and Signal Protocol trade-offs

MLS and the Signal Protocol also differ on ordering, deniability, post-quantum work and standardisation.

Ordering. The Double Ratchet tolerates lost and out-of-order messages by numbering them and keeping skipped keys for a while. MLS also tolerates out-of-order application messages, but the group has a linear history of epochs, so everyone must apply the same Commit. RFC 9750 explains that the Delivery Service usually orders Commits, and that two members committing at once is a realistic risk. It frames the choice of ordering strategy in terms of consistency and partition tolerance, which matters on links that are slow or often split.

Deniability. X3DH is designed so that neither party gets a publishable cryptographic proof of what was said or that they talked. RFC 9750 says MLS makes no claims about deniability and supports deployments where a recipient can prove who sent a message, for example to report abuse.

Post-quantum work. PQXDH adds post-quantum forward secrecy to the initial key agreement, and the current revision of the Double Ratchet specification adds a Sparse Post-Quantum Ratchet and a Triple Ratchet that combines it with the original. The cipher suites defined in RFC 9420 use elliptic-curve Diffie-Hellman key encapsulation.

Standardisation. MLS is an IETF Standards Track protocol with a cipher suite registry for interoperable implementations. The Signal specifications are published by Signal and placed in the public domain.

Frequently asked questions

Is MLS a replacement for the Signal Protocol?

Not directly. They were designed for different starting points. The Signal specifications describe two-party sessions, while MLS defines group key agreement. Which suits an application depends on group sizes, how changes are ordered, and properties such as deniability.

Can MLS be used for one-to-one chats?

Yes. RFC 9420 covers groups ranging from two members to thousands, so a two-person conversation is a group of two.

Which is deniable, MLS or the Signal Protocol?

X3DH is designed so that neither party gets a publishable cryptographic proof of what was said or that they talked. RFC 9750 says MLS makes no claims about deniability and supports deployments where a recipient can prove who sent a message, for example to report abuse.

Is the Signal Protocol post-quantum?

Partly. PQXDH adds post-quantum forward secrecy to the initial key agreement while mutual authentication still relies on the hardness of the discrete log problem, and the current Double Ratchet specification adds a Sparse Post-Quantum Ratchet and a Triple Ratchet.

Why do sender keys scale poorly for groups?

RFC 9420 says recovering from a compromise with sender keys needs a number of key update messages that grows with the square of the group size. An attacker who learns a sender key can often keep reading that member's messages.

Does MLS handle out-of-order messages?

It tolerates out-of-order application messages, but the group has a linear history of epochs, so every member must apply the same Commit. The Double Ratchet instead numbers messages and keeps skipped keys for a while.

Sources

Build it with Offline Protocol

The Offline Protocol SDK uses MLS for application messages and group communication. The security page explains what that covers, which control traffic is signed plaintext instead, and the residual risks recorded in its threat model.

Read the security model