NATS Account Storage Quota Approaching Limit: Prevent Stream Write Failures

NATS accounts have a configurable JetStream storage quota that caps how much disk (or memory) storage the account’s streams can reserve. When an account’s total stream reservations approach this quota, the account is nearing the point where new stream creates and writes to existing streams will fail. This check flags accounts where JetStream storage reservations are approaching the configured limit.

Why this matters

NATS enforces storage quotas by reservation, not by actual bytes on disk. When you create a stream with max_bytes: 1GB and 3 replicas, NATS reserves 3 GB against the account quota — regardless of whether the stream currently holds 10 KB or 900 MB of actual data. This means an account can exhaust its quota with mostly empty streams.

When the quota is reached, the impact is immediate and disruptive. Every stream create call fails. Every stream update that increases max_bytes fails. And critically, writes to existing streams that don’t have max_bytes set (unlimited streams) fail because the server can’t guarantee storage availability. The error surfaces as a JetStream API error: insufficient storage resources available.

The failure cascades through the application layer. Producers get publish errors and must implement retry logic or drop messages. Services that create streams on demand — a common pattern for per-tenant or per-workflow isolation — can’t provision new streams. Consumers continue reading existing data, but the system is effectively read-only for JetStream workloads.

This is especially dangerous in multi-tenant deployments where each tenant maps to a NATS account. One tenant’s storage sprawl doesn’t affect other tenants directly, but within that tenant’s account, every JetStream operation is impacted. And because the quota is based on reservations rather than actual usage, the problem can appear suddenly — a single large stream create can push the account over the edge even if actual disk usage is modest.

Common causes

Streams created without max_bytes. Streams without explicit byte limits reserve storage based on the remaining account quota. Each unlimited stream effectively claims “as much as I might need,” rapidly consuming the reservation budget. This is the most common cause.
Over-replicated streams. A stream with num_replicas: 3 reserves three times its max_bytes against the account quota. If the deployment doesn’t need R3 for every stream, the excess replicas waste reservation capacity.
Stale or unused streams. Streams created for testing, one-off migrations, or deprecated features continue to hold their storage reservations. Without periodic cleanup, these accumulate.
Large max_bytes with low actual usage. A stream configured with max_bytes: 100GB but holding only 2 GB of data still reserves the full 100 GB. Overly generous limits waste quota.
Account quota not scaled with growth. The original quota was sized for the initial deployment. As the number of streams, services, and data retention requirements grew, the quota wasn’t updated to match.
Reservation multiplier misunderstanding. Operators set max_bytes thinking it represents total storage, not per-replica storage. A 10 GB stream with R3 reserves 30 GB, not 10 GB.

How to diagnose

Check account storage usage vs. quota

nats server account info <account_name>

This shows the account’s JetStream storage limit, current reservation, and actual usage. Compare the storage reserved value against the storage limit.

List streams sorted by reservation size

nats stream list  # account is selected via NATS context/credentials

Review each stream’s max_bytes and replica count. The reservation for each stream is max_bytes × num_replicas. Streams without max_bytes set show as unlimited and are the most problematic.

For a detailed breakdown:

nats stream report  # account is selected via NATS context/credentials

This shows actual bytes stored versus configured limits per stream, making it easy to spot overprovisioned streams.

Identify streams without byte limits

nats stream list --json | jq '.[] | select(.config.max_bytes == -1) | .config.name'

These unlimited streams are the primary targets for optimization since they consume unbounded reservation space.

Calculate reservation waste

1
package main
2

3
import (
4
    "context"
5
    "fmt"
6
    "log"
7

8
    "github.com/nats-io/nats.go"
9
    "github.com/nats-io/nats.go/jetstream"
10
)
11

12
func main() {
13
    nc, err := nats.Connect("nats://localhost:4222")
14
    if err != nil {
15
        log.Fatal(err)
16
    }
17
    defer nc.Close()
18

19
    js, err := jetstream.New(nc)
20
    if err != nil {
21
        log.Fatal(err)
22
    }
23

24
    ctx := context.Background()
25
    var totalReserved, totalUsed int64
26

27
    lister := js.ListStreams(ctx)
28
    for info := range lister.Info() {
29
        reserved := info.Config.MaxBytes * int64(info.Config.Replicas)
30
        used := int64(info.State.Bytes) * int64(info.Config.Replicas)
31
        waste := reserved - used
32
        if info.Config.MaxBytes > 0 {
33
            fmt.Printf("%-30s reserved=%-12d used=%-12d waste=%-12d replicas=%d\n",
34
                info.Config.Name, reserved, used, waste, info.Config.Replicas)
35
            totalReserved += reserved
36
            totalUsed += used
37
        } else {
38
            fmt.Printf("%-30s UNLIMITED replicas=%d\n",
39
                info.Config.Name, info.Config.Replicas)
40
        }
41
    }
42
    fmt.Printf("\nTotal reserved: %d bytes, Total used: %d bytes, Waste: %d bytes\n",
43
        totalReserved, totalUsed, totalReserved-totalUsed)
44
}

How to fix it

Immediate: free up reservation space

Set max_bytes on unlimited streams. This is the highest-impact change. Determine reasonable byte limits based on actual usage plus growth headroom and apply them:

nats stream edit <stream_name> --max-bytes 10GB

For each unlimited stream, check current usage with nats stream info <stream_name> and set max_bytes to 2–5x current actual usage, depending on growth expectations.

Delete unused streams. Streams from testing, deprecated features, or one-off data migrations should be removed:

nats stream delete <stream_name>

Each deletion immediately frees the full reservation (max_bytes × replicas) from the account quota.

Purge stale data from oversized streams. If a stream has accumulated historical data beyond its useful retention:

nats stream purge <stream_name> --keep 1000  # keep latest 1000 messages

This reduces actual storage usage but only frees reservation space if combined with lowering max_bytes.

Short-term: optimize reservations

Right-size max_bytes on overprovisioned streams. A stream with max_bytes: 100GB but only 5 GB of actual data wastes 95 GB of reservation. Lower the limit:

nats stream edit <stream_name> --max-bytes 15GB

Use actual usage as a baseline and add a reasonable buffer for growth and retention.

Reduce replica count where R3 isn’t needed. Development streams, ephemeral work queues, and non-critical logging streams may not need three replicas:

nats stream edit <stream_name> --replicas 1

This reduces the reservation by two-thirds for that stream.

Enable S2 compression. Compression reduces actual on-disk size, which may allow you to lower max_bytes without risk:

nats stream edit <stream_name> --compression s2

Add max_age to expire old messages. Time-based retention prevents unbounded growth and makes it safer to set tighter max_bytes limits:

nats stream edit <stream_name> --max-age 7d

Long-term: prevent quota creep

Increase the account quota if the workload justifies it. In operator/JWT mode:

nsc edit account <account_name> --js-disk-storage 500GB
nsc push -a <account_name>

In server config:

1
accounts {
2
  MY_ACCOUNT {
3
    jetstream {
4
      max_store: 500GB
5
    }
6
  }
7
}

Enforce max_bytes as a stream creation policy. Use account-level or operator-level policies to reject stream creates that don’t include max_bytes. This prevents future unlimited streams from eroding the quota.

Monitor reservation utilization proactively. Alert when reservation approaches the threshold.

Synadia Insights evaluates this automatically at every collection interval, catching quota creep before it causes write failures.

Frequently asked questions

Why is my quota full when actual disk usage is low?

NATS enforces quotas by reservation (max_bytes × replicas), not actual bytes stored. A stream with max_bytes: 50GB and R3 reserves 150 GB of quota even if it currently stores 1 GB. To reclaim quota space, lower max_bytes to match actual needs.

What happens when the quota is reached?

New stream creates fail, stream updates that increase max_bytes fail, and writes to unlimited streams fail. The error message is insufficient storage resources available. Existing streams with max_bytes already set continue to accept writes until they individually reach their byte limit.

Does purging a stream free quota?

Only indirectly. Purging removes messages and frees disk space, but the stream’s max_bytes reservation remains unchanged. To free quota, you must either lower max_bytes or delete the stream entirely.

How is the reservation calculated for replicated streams?

The reservation is max_bytes × num_replicas. A stream with max_bytes: 10GB and num_replicas: 3 reserves 30 GB against the account quota. This ensures the cluster has enough capacity across all replicas.

Can I set different storage quotas per stream?

Not directly at the account level. The account quota is a single aggregate limit. However, you control each stream’s reservation individually via max_bytes. Setting appropriate max_bytes on every stream is the primary mechanism for managing how the account quota is distributed.

FEATURED

RESOURCES

Comparisons