Should You Use NATS KV or Object Store for Many Small Objects?

If you have many small values and many application replicas read all of them at startup, start by evaluating NATS Key/Value (KV), not Object Store. Object Store is usually a better fit for file-like payloads that are fetched on demand, while KV Get() is closer to a direct lookup of the latest value for a key.

This guidance comes from a real NATS community question about a bucket with many mostly-small objects and many pods fetching every object during startup.

Short answer: should small blobs use NATS KV or Object Store?

Use KV for the common path when values are small, lookup-oriented, and read frequently by many replicas. Use Object Store for payloads that are truly object-like, large enough to benefit from chunking, or fetched selectively on demand.

For this pattern, consider:

Keeping small values in KV, with an appropriate maximum value size for the bucket and your JetStream/account limits.
Using Object Store only for values that are truly file-like or too large for your KV design.
Separating storage from notifications: use a stream or KV update mechanism to notify pods about changes, and fetch object data only when required.
Avoiding one consumer or subscription per object. Prefer one or a small number of consumers per pod, optionally using subject filters and subject hierarchy to group updates.

Why can Object Store reads create consumer churn?

NATS Object Store is built on JetStream. Objects are stored as chunks, and an Object Store Get() streams those chunks back to the client. To do that, the client creates an ephemeral ordered consumer for the read.

That behavior is useful for object retrieval because it gives the client ordered streaming delivery of the object chunks. However, it also means that a workload like this can create a large amount of short-lived consumer activity:

10,000 objects in a bucket
100 application pods
every pod fetching every object at startup

Even if most objects have only one chunk, the access pattern still behaves like many object-stream reads. The practical result can be memory and CPU pressure on the NATS servers due to rapid consumer creation and cleanup, plus the data transfer itself.

KV reads are different. A KV Get() is a direct lookup of the latest value for a key and does not create a consumer for each read in the same way. That difference is why a workload that was comfortable on KV can become expensive after moving the same shape of data to Object Store.

Does a one-chunk object avoid the Object Store consumer?

No. Do not plan around one-chunk objects as an optimization. Object Store is designed around streaming object chunks, and the read path uses a consumer even when the object is small enough to fit in one chunk.

If most of your data is small and you expect frequent full-bucket reads across many replicas, reconsider whether Object Store is the right primary abstraction for those values.

How should storage and notifications be separated?

A common design mistake is to use the storage mechanism as the notification mechanism. These are different concerns:

Storage: Where the current data lives.
Notification: How applications learn that something changed.

Object Store is a good place to store and retrieve file-like payloads. It is not usually the best mechanism for every pod to maintain many per-object subscriptions or to re-download every object aggressively.

A more scalable design is usually:

Store small values or manifests in KV.
Store large blobs in Object Store when needed.
Publish update notifications to a separate stream or maintain update state in KV.
Have each pod consume notifications using one consumer, or a small bounded number of consumers.
Use subject hierarchy to group files or keys so consumers can filter by meaningful categories instead of by every individual object.

For example, instead of creating per-object consumers, group updates by application-level dimensions such as tenant, namespace, object type, or shard. The exact subject design should match your access pattern, but the goal is to avoid 10,000 separate consumers when a smaller number of filtered consumers can express the same work.

When is NATS KV the better fit?

KV is often a better fit when:

Values are usually small.
Callers frequently read individual keys.
Many replicas need the latest value.
The application does not need to stream large object chunks.
You want a lookup-style read path rather than a streaming object read path.

If only a small fraction of values exceed your current KV size setting, increasing the KV maximum value size may be the simpler and more appropriate option. Before doing that, review the practical limits for your deployment, including JetStream storage, account limits, client memory behavior, and the worst-case value sizes you expect.

When is Object Store the better fit?

Object Store is usually the better fit when:

Payloads are file-like blobs.
Objects may be large enough to benefit from chunked storage and streaming retrieval.
Applications fetch specific objects on demand rather than reading the entire bucket on every startup.
The application can tolerate object retrieval being a streaming operation.

Object Store is not wrong for small objects, but it can be the wrong fit for a workload that treats many small objects like a high-fanout configuration database.

What if most values are small but a few are large?

Use a hybrid design:

Store normal-sized values in KV.
Store large payloads in Object Store.
Store metadata or a pointer in KV that tells the application where to fetch the large payload.
Send update notifications separately, so consumers know what changed without subscribing to every object individually.

This lets the common path stay efficient while still supporting larger data when needed.

How can you reduce startup load when every pod must initialize state?

If every pod must initialize local state from NATS, reduce the startup spike:

Limit concurrent Object Store Get() calls per pod.
Add jitter so many pods do not fetch the same data at exactly the same time.
Cache data locally when the deployment model allows it.
Use a manifest or index so pods can fetch only objects that changed.
Prefer one or a few update consumers per pod instead of one consumer per object.
Consider whether all pods really need all objects, or whether the data can be partitioned by subject or application shard.

These measures help, but they do not change the core tradeoff: Object Store reads are streaming object reads, not KV-style direct lookups.

Practical recommendation

For a bucket with around 10,000 mostly-small values and around 100 pods that read everything on startup, start by evaluating KV with a larger maximum value size. If the values are mostly configuration-like or lookup-oriented, KV is likely closer to the intended access pattern.

Use Object Store for the subset of data that is truly object-like or too large for the KV design. Then use a separate stream or KV-based update mechanism for notifications, with a small number of consumers per pod and subject filters where helpful.

The key design principle is to avoid coupling every stored item to its own consumer. Keep storage, lookup, and notification responsibilities separate, and choose KV or Object Store based on how the data is read most of the time.

Want help from the NATS experts? Meet with our architects to get help tailored to your use case and environment.