Scaling NATS for Per-User Real-Time Notifications

A common community question is whether a NATS cluster can scale to hundreds of thousands or even a million concurrent clients when each client needs unique real-time notifications and some amount of missed-message catch-up.

The short answer is: design around the whole workload, not just the connection count. A million connected clients with lightweight Core NATS subscriptions is a different problem from a million long-lived persistent JetStream consumers. For per-user notifications, the main scaling pressure is often the number of active catch-up consumers, reconnect rate, stream layout, and storage I/O—not simply the number of TCP connections.

Clarify what “one queue per client” means

In NATS, a “queue” usually means a queue subscription: multiple subscribers in the same queue group share work, and each message is delivered to one member of the group. That is usually not what you want for per-user notifications.

For unique notifications, a more typical model is:

• publish each user or device notification to a subject that includes a user/device identifier, or a hash plus identifier;
• persist those messages in JetStream if clients need catch-up;
• let connected clients receive live messages through ordinary subscriptions;
• use JetStream only when a client needs to recover messages missed while disconnected.

The important distinction is that a persisted mailbox-like experience does not necessarily require one durable JetStream consumer per client.

Watch the reconnect and catch-up pattern

If clients reconnect frequently, catch-up behavior can dominate the design.

For example, with 1,000,000 devices reconnecting every 15 minutes on average, the average reconnect rate is roughly 1,100 reconnects per second. The number of concurrent catch-up operations depends on how long each catch-up takes. If catch-up takes 20 seconds on average, that is over 20,000 concurrent catch-up operations. If reconnects are bursty, the peak can be much higher.

That does not automatically make the design impossible, but it changes what you should optimize:

• avoid creating a long-lived durable JetStream consumer for every connected client;
• keep catch-up consumers short-lived;
• shard persisted streams so load is distributed across the cluster;
• use live Core NATS subscriptions after the client is caught up;
• store enough client-side or application-side position state to know where to resume.

A scalable pattern: sharded streams plus short-lived catch-up

A practical design is to separate “catch-up” from “live delivery.”

1. Publish to sharded notification subjects

Instead of putting every notification into one stream, add a deterministic shard token to the subject. For example:

1
notifications.<shard>.<client_id>

Where <shard> is computed from the client ID, such as a hash modulo the number of shards:

1
notifications.1.user-123
2
notifications.7.user-456
3
notifications.19.user-789

Then create multiple streams, each responsible for a shard or group of shards, for example:

1
notifications.1.>
2
notifications.2.>
3
...
4
notifications.20.>

The exact number of shards is a capacity-planning decision. Starting with multiple shards gives you more options to distribute storage, replicas, and read load later. Some deployments can use NATS subject mapping features for deterministic partitioning; others compute the shard token in the publishing service. The important property is that the same client ID always maps to the same shard unless you deliberately rebalance.

2. Use JetStream for persistence, not for every live subscription

JetStream should store the messages needed for offline catch-up. However, once a client is current, it does not necessarily need to remain attached to a JetStream consumer.

A common approach is:

1. client connects and provides its last received position;
2. service creates or uses a short-lived ordered consumer filtered to that client’s subject;
3. client receives missed messages;
4. once caught up, the catch-up consumer is no longer needed;
5. live messages arrive through a Core NATS subscription.

Ordered consumers are useful here because they are lightweight and fit short-lived sequential reads. They are not a substitute for capacity testing, but they are generally a better fit than maintaining a durable consumer for every user or device.

3. Use stream republish for live delivery

A JetStream stream can be configured to republish each message—as it is stored—onto a Core NATS subject. Republish is a live, real-time mechanism: it mirrors new messages as they are ingested into the stream, rather than replaying historical messages on demand. With this pattern, clients subscribe to the republished live subject after they are caught up.

Conceptually:

1
published notification -> JetStream stream -> republished live subject -> connected client

This lets the stream remain the durable source of truth while live delivery uses normal NATS subscription mechanics. That avoids tying every online client to a long-lived JetStream consumer.

4. Detect gaps and backfill when needed

In distributed systems, clients can disconnect between catch-up and live delivery, or miss a live message during a network transition. Plan for this explicitly.

Republished JetStream messages carry stream metadata in headers, including Nats-Stream, Nats-Sequence, and Nats-Last-Sequence. The Nats-Last-Sequence header lets a subscriber compare each message against the last sequence it processed, so it can detect when messages were missed. If a client sees a gap, it can backfill from JetStream rather than staying permanently attached to a consumer.

For small gaps, JetStream’s direct get can be useful because it fetches messages straight from the stream without creating a consumer; this does require the stream to have direct access enabled. For larger gaps, a short-lived ordered consumer may be the better option. Which choice is best depends on message volume, expected gap size, and how much position state your client or gateway keeps.

Retention still needs a clear policy

If messages only need to exist until clients have caught up, define what that means operationally.

With the sharded stream and short-lived consumer pattern, stream retention is usually governed by limits such as time, bytes, or message count. Acknowledging a message in a temporary catch-up consumer does not necessarily mean the message is immediately deleted from the stream for everyone. That is often acceptable for notifications, as long as the stream retains enough history to cover the offline window you support.

If you need strict per-client deletion immediately after receipt, that introduces more per-client state and may push you toward a different design. In many notification systems, a simpler approach is to retain recent history for a bounded period and have each client or an application service track the last received sequence.

Shard from day one if you expect horizontal growth

A single large stream can become a scaling constraint because stream writes, storage, replication, and reads are concentrated around that stream’s placement. Sharding streams gives you more control over where load lands.

For example, with several NATS servers grouped by placement tags, you can distribute stream replicas across groups so different shards use different server sets. Writes for a shard go to the replicas for that shard, and reads such as ordered-consumer catch-up or direct gets are served by the nodes that host that stream.

This does not remove the need to size the cluster, storage, and network carefully, but it keeps the data layer more horizontally scalable than a single-stream design.

What to test before committing to the architecture

Before assuming a design will handle 100K, 500K, or 1M clients, test with realistic workload variables:

• concurrent connected clients;
• reconnect rate and burstiness;
• average and peak missed-message count per reconnect;
• number of active catch-up consumers;
• message size;
• publish rate per shard;
• stream replica count and storage type;
• retention window;
• direct-get rate for gap repair;
• client subscription fanout;
• gateway or load balancer behavior during reconnect storms.

Connection count alone is not enough to model the system. A workload with 1M mostly idle live subscribers can behave very differently from 1M clients repeatedly reconnecting and replaying history.

Practical guidance

For a large per-user notification system on NATS:

• avoid modeling each user as a long-lived persistent queue;
• use subject-based addressing for per-user messages;
• persist notifications in JetStream only for the catch-up window you need;
• shard streams by a deterministic client or user hash;
• use short-lived ordered consumers for catch-up;
• switch to Core NATS subscriptions, potentially via stream republish, for live delivery;
• use sequence metadata to detect gaps and backfill;
• capacity test reconnect and catch-up behavior, not just steady-state connections.

The architecture can scale much more cleanly when JetStream is used for bounded recovery and Core NATS is used for live fanout. The key is to minimize long-lived per-client server-side state, distribute persisted data across shards, and make missed-message recovery explicit.

Want help from the NATS experts? Meet with our architects to get help tailored to your use case and environment.