Can JetStream Pull Consumers Fetch Strictly Ordered Batches?

A common community question is whether a JetStream pull consumer can fetch the next N messages as one strictly ordered batch while preventing other application instances from receiving messages in the meantime.

The short answer: not by using Fetch(N) alone when multiple clients are bound to the same pull consumer. A pull request for N messages is not an atomic reservation of the next N stream or consumer sequence numbers. If several clients are fetching from the same consumer, JetStream may distribute messages across those waiting pull requests.

If you need strict batch processing, design for one active batch processor per ordered sequence. Depending on your NATS Server version and operational requirements, that may mean running one instance, using consumer pinning, using an application-level lease, or partitioning the workload.

Consumer ordering versus atomic batch reservation

JetStream has a server-side entity called a consumer. Your services or application instances are clients that bind to that consumer and request messages.

A pull consumer can deliver messages in order for the consumer, but this does not mean that a single Fetch(10) call from one client gets an exclusive, consecutive range if another client is also fetching. With multiple clients using the same consumer, concurrent pull requests are independent. The server can distribute messages among them.

For example, if two application instances both fetch from the same pull consumer, one instance may receive messages with payloads like:

1
Message 1
2
Message 2
3
Message 4
4
Message 6
5
...

while another receives:

1
Message 3
2
Message 5
3
Message 7
4
...

Each client still receives its own messages in consumer order, but the overall sequence is spread across the clients. The exact interleaving depends on timing and ack behavior. The important point is that a fetch batch is not a lock on the next N messages.

What MaxAckPending does and does not do

MaxAckPending limits how many delivered messages may remain unacknowledged for a consumer. It is useful for backpressure and for limiting in-flight work.

It does not turn a fetch request into an atomic batch reservation.

There are two common surprises here:

1. MaxAckPending=1 limits concurrency, but also prevents collecting an unacked batch. If only one unacked message may be outstanding, a client cannot receive a batch of 10 messages and hold all 10 unacked until the batch succeeds.
2. A delayed negative acknowledgment is not the same as pausing the ordered stream. If a message is NAKed with a delay, that message is scheduled for later redelivery. Later messages may be delivered before the delayed redelivery occurs, depending on the consumer configuration and client activity.

If your goal is “do not process anything after this message until this same message succeeds,” then NakWithDelay is usually the wrong flow-control mechanism. You likely need to stop pulling, keep work serialized, or design the failed operation to be retried without allowing later side effects to overtake it.

If you need a strict ordered batch, use one active fetcher

For a workload such as “take the next N messages, write them as one batch to an external system, and only then acknowledge progress,” you need one active processor for that ordered sequence.

Options include:

• Run exactly one instance of that processor.
• Move the batcher into a separate service that is deployed as a singleton.
• Use a distributed lease or lock with a TTL, such as one implemented with NATS KV.
• On NATS Server 2.11 or later, consider consumer pinning so only one client actively receives messages from a consumer while others stand by.

Consumer pinning was introduced to support the “single active client with standby clients” pattern without requiring a separate leader-election system. See the NATS Server 2.11 release notes for details: Pinning a consumer to a specific client.

In the Go client’s newer JetStream API, the relevant consumer configuration includes fields such as:

1
PriorityPolicy:  jetstream.PriorityPolicyPinned,
2
PinnedTTL:       30 * time.Second, // illustrative; size it to comfortably exceed your batch processing time
3
PriorityGroups:  []string{"batcher"},

When fetching or consuming, the client also needs to use the matching priority group. The exact call shape depends on the client API you are using, so check your client library documentation and tests for the supported options.

Pinning is a coordination mechanism, not a substitute for safe processing

Consumer pinning is generally simpler than building your own lease for this use case, but it has limits. Pinning only coordinates clients that bind to the consumer using the same priority group; a client that consumes from the same consumer without requesting that group is not held in standby, so every participating client must opt in. Pinning is also not a strict consensus guarantee — it has roughly the same properties as a TTL-based lease, so you should still design the processor to tolerate retries and failover. TTL-based ownership mechanisms depend on timeouts: if a process stalls, loses connectivity, or crashes at an awkward point, another process may eventually take over.

That is usually what you want for availability, but it means your external writes should be idempotent or otherwise safe to retry.

What about AckAll for batch acknowledgment?

JetStream’s AckAll policy can be useful for batch-style progress tracking because acknowledging the last message can acknowledge earlier pending messages for the consumer.

However, use it carefully:

• It is only appropriate when you understand exactly which messages are pending for that consumer.
• It is safest with one active client pulling from the consumer.
• With multiple active clients, AckAll can acknowledge messages beyond the local batch you think you are committing.
• If your service writes the batch to an external system and crashes before the ack reaches the server, messages can be redelivered. The external write path must handle duplicates or be idempotent.

A typical single-active-client pattern is:

1
1. Fetch up to N messages.
2
2. Build the external batch.
3
3. Write the batch successfully.
4
4. Ack progress only after the write succeeds.

With AckAll, step 4 may be implemented by acknowledging the final message in the batch. With explicit acking, you would ack individual messages after success. In both cases, a crash between the external write and the ack can produce redelivery, so the sink must be safe for replay.

If the batch must be atomic, consider making it one message

If the application-level requirement is truly “these records succeed or fail as one unit,” the cleanest model may be to publish them as one JetStream message whose payload contains the batch.

That changes the acknowledgment unit to match the business unit of work:

• One message represents one batch.
• One ack represents successful processing of that batch.
• Redelivery replays the whole batch.

This is not always possible, especially when many independent producers are writing individual events, but it is worth considering when the downstream operation is inherently batch-oriented.

Scaling while preserving order: partition the stream of work

Strict total ordering and high parallelism are in tension. If every message must be processed in one global order, only one processor can safely advance that order.

To scale throughput, partition the workload so each partition has its own ordered sequence. Common approaches include:

• Partition by tenant, account, customer, aggregate ID, or another stable key.
• Use separate subjects or streams per partitioning scheme.
• Run one active processor per partition.
• Preserve order within a partition, while allowing different partitions to process in parallel.

This gives you concurrency without pretending that a single global ordered log can be processed by many workers at once without coordination.

Practical guidance

For strict ordered batch processing with JetStream pull consumers:

1. Do not assume Fetch(N) reserves the next N consecutive messages when multiple clients are fetching.
2. Use MaxAckPending for in-flight limits, not as a batch mutex.
3. Avoid using delayed NAK as a way to pause all later processing.
4. Use one active fetcher per ordered sequence.
5. Consider consumer pinning on NATS Server 2.11+ for active/standby processing.
6. Consider AckAll only when a single active client owns the consumer’s pending work and you have thought through crash and redelivery behavior.
7. Make downstream writes idempotent, because redelivery is part of the reliability model.
8. Partition the workload if you need both ordering and throughput.

The core design choice is simple: if you need a strict ordered batch, make sure only one active processor can advance that ordered sequence. If you need more throughput, split the workload into multiple ordered sequences and process those partitions independently.

Want help from the NATS experts? Meet with our architects to get help tailored to your use case and environment.