NATS Pinned Consumer Policy Mismatch: Detecting Misconfigured Pinned Consumers

A pinned consumer policy mismatch fires when a JetStream consumer is marked for pinned monitoring — its metadata has both io.nats.monitor.enabled and io.nats.monitor.pinned set to true — but its priority_policy is not pinned_client. The metadata says the operator intends one client at a time to own the consumer, with the others standing by. The configuration doesn’t enforce that, so the declared client pinning is not in effect and every pulling client receives messages.

Why this matters

Pinned clients are a pattern for workloads that need one active processor with automatic failover. Under the pinned_client priority policy (NATS Server 2.11.0+), the server pins the first client that pulls for a priority group: it assigns a pin ID, returns it in the Nats-Pin-Id header, and from then on serves only pull requests that carry that ID. Other clients in the group keep their pull requests waiting. If the pinned client stops pulling for longer than the consumer’s priority_timeout (two minutes by default), or an operator unpins it, the server pins the next waiting client and publishes an advisory on $JS.EVENT.ADVISORY.CONSUMER.UNPINNED / PINNED.

The pinned_client policy is the mechanism that enforces this. Without it, the consumer is an ordinary pull consumer and distributes messages across every client that pulls. This means:

• Single-processor assumptions break. Messages meant to flow through one client are spread across all of them, so per-client processing order and any in-memory state that depends on seeing every message (session state, aggregation windows, sequence tracking) no longer hold.
• There is no primary and no standby. All clients are peers. The failover that pinning provides doesn’t exist, because nothing was pinned in the first place.
• Health checks that rely on pinning fail. The same io.nats.monitor.pinned metadata drives nats server check consumer, which reports critical (“Not pinned client priority mode”) for a consumer that isn’t using pinned_client, and also when a configured group has no pinned client.

The metadata records intent; the priority policy enforces it. When the two disagree, the system behaves differently from what the operator expects, and the resulting processing errors are hard to diagnose after the fact.

This check only looks at consumers whose leader runs NATS Server 2.11.0 or later, because older servers don’t report priority_policy. Consumers with invalid monitor metadata are skipped here and reported by Invalid Consumer Monitor Metadata (consumer-020) instead.

Common causes

• The wrong priority policy was chosen. overflow, prioritized and pinned_client are three different policies. overflow adds spillover capacity under backlog and prioritized (2.12+) ranks clients; neither pins a single client. A consumer marked as pinned but created with --overflow-groups or --prioritized-groups fires this check.

• Consumer created before priority policies existed. The operator added the monitor metadata to mark intent while the servers were older than 2.11.0. After the upgrade, the consumer was never updated to set the policy.

• Configuration template missing the priority fields. Infrastructure-as-code templates (Terraform, Helm, Pulumi) define the consumer with metadata but omit priority_policy and priority_groups. Every consumer created from the template has the mismatch.

• Manual consumer creation with incomplete configuration. An operator creates a consumer via the CLI or API and adds the metadata but not the policy. The metadata is a monitoring declaration; it doesn’t configure delivery.

• Environment-specific configuration drift. The consumer is correctly configured in production, but a staging or development copy lacks the policy. The metadata matches across environments while the behavior differs.

How to diagnose

Check consumer configuration

nats consumer info STREAM_NAME CONSUMER_NAME -j | \
  jq '{metadata: .config.metadata, priority_policy: .config.priority_policy, priority_groups: .config.priority_groups}'

If the metadata shows io.nats.monitor.enabled and io.nats.monitor.pinned as true but priority_policy is absent, none, overflow or prioritized, the mismatch is confirmed.

On a correctly configured consumer, the same info response also shows who holds the pin: each entry under .priority_groups in the consumer state carries a pinned_client_id once a client is pinned.

List all consumers marked as pinned

# Find all consumers across streams that declare pinned monitoring
for stream in $(nats stream list -n); do
  for consumer in $(nats consumer list "$stream" -n); do
    result=$(nats consumer info "$stream" "$consumer" -j 2>/dev/null | \
      jq -r 'select(.config.metadata["io.nats.monitor.pinned"] == "true") |
        "\(.stream_name)/\(.name): priority_policy=\(.config.priority_policy // "NOT SET")"')
    [ -n "$result" ] && echo "$result"
  done
done

Anything other than priority_policy=pinned_client in the output is a mismatch.

Verify server version supports priority_policy

nats server list

The output includes a Version column for each server (you can also use nats server info <name>). The pinned_client policy requires NATS Server 2.11.0 or later. On older servers the policy cannot be set even if the metadata is present.

Programmatic detection

1
// Go: detect pinned metadata without the pinned_client policy.
2
// PriorityPolicy lives in github.com/nats-io/nats.go/jetstream — the
3
// legacy nats.JetStreamContext API does not expose it.
4
import (
5
    "context"
6
    "fmt"
7

8
    "github.com/nats-io/nats.go"
9
    "github.com/nats-io/nats.go/jetstream"
10
)
11

12
ctx := context.Background()
13
js, _ := jetstream.New(nc)
14
stream, _ := js.Stream(ctx, "STREAM_NAME")
15

16
consumers := stream.ListConsumers(ctx)
17
for ci := range consumers.Info() {
18
    md := ci.Config.Metadata
19
    marked := md["io.nats.monitor.enabled"] == "true" && md["io.nats.monitor.pinned"] == "true"
20
    if marked && ci.Config.PriorityPolicy != jetstream.PriorityPolicyPinned {
21
        fmt.Printf("MISMATCH: %s/%s is marked pinned but priority_policy=%v\n",
22
            ci.Stream, ci.Name, ci.Config.PriorityPolicy)
23
    }
24
}

1
# Python: detect pinned consumer policy mismatch.
2
# Requires the new nats-jetstream package — the legacy nats-py client's
3
# ConsumerConfig does not expose priority_policy at all, so it cannot
4
# read the field. (With the legacy client, parse the raw JSON from
5
# $JS.API.CONSUMER.INFO instead.)
6
from nats import client, jetstream
7

8
nc = await client.connect("nats://localhost:4222")
9
js = jetstream.new(nc)
10

11
async for ci in js.list_consumers("STREAM_NAME"):
12
    metadata = ci.config.metadata or {}
13
    marked = (metadata.get("io.nats.monitor.enabled") == "true"
14
              and metadata.get("io.nats.monitor.pinned") == "true")
15
    if marked and ci.config.priority_policy != "pinned_client":
16
        print(f"MISMATCH: {ci.stream_name}/{ci.name} "
17
              f"is marked pinned but priority_policy={ci.config.priority_policy}")

How to fix it

Set the pinned_client priority policy

At least one priority group is required whenever priority_policy is set — the server rejects a policy without a group (error 10159), so the group is not an optional add-on. Priority groups also require a pull consumer.

New or recreated consumers. nats consumer add exposes the policy directly. --pinned-groups sets priority_policy=pinned_client with the named groups, and --pinned-ttl sets how long the pinned client may go without pulling before it loses the pin:

nats consumer add STREAM_NAME CONSUMER_NAME --pull --defaults \
  --pinned-groups workers --pinned-ttl 30s \
  --metadata io.nats.monitor.enabled=true \
  --metadata io.nats.monitor.pinned=true

Existing consumers. These flags exist only on add; nats consumer edit has no priority flags. Apply the change through a config file instead:

nats consumer info STREAM_NAME CONSUMER_NAME -j | \
  jq '.config + {priority_policy: "pinned_client", priority_groups: ["workers"]}' > consumer.json
nats consumer edit STREAM_NAME CONSUMER_NAME --config consumer.json

Alternatively, run nats consumer edit STREAM_NAME CONSUMER_NAME --interactive and add both fields in the editor. Set priority_timeout in the same file if the two-minute default doesn’t suit the workload.

Changing the policy is only half the job: clients must name the group on their pull requests. A pull request that doesn’t name a group is rejected with 400 Bad Request - Priority Group missing, and one that names a group the consumer doesn’t have gets 400 Bad Request - Invalid Priority Group. Roll out the client change together with the consumer change.

Verify the fix

After updating, confirm the configuration is correct:

nats consumer info STREAM_NAME CONSUMER_NAME -j | \
  jq '{metadata: .config.metadata, priority_policy: .config.priority_policy, priority_groups: .config.priority_groups}'

The monitor metadata and priority_policy: "pinned_client" should both be present. To confirm pinning end to end, run the upstream health check, which reads the same metadata:

nats server check consumer --stream STREAM_NAME --consumer CONSUMER_NAME

It reports OK with the number of pinned clients once a client has pulled and been pinned.

Update infrastructure-as-code templates

If consumers are managed via Terraform, Helm, or other IaC tools, update the template to include the priority policy:

1
# Terraform example
2
resource "jetstream_consumer" "pinned_processor" {
3
  stream_id = jetstream_stream.orders.id
4

5
  durable_name     = "pinned-processor"
6
  ack_policy       = "explicit"
7
  priority_policy  = "pinned_client"
8
  priority_groups  = ["workers"]
9
  priority_timeout = 30
10

11
  metadata = {
12
    "io.nats.monitor.enabled" = "true"
13
    "io.nats.monitor.pinned"  = "true"
14
  }
15
}

In the Terraform provider, priority_policy, priority_groups and priority_timeout all force a new resource, so applying this change deletes and recreates the consumer. Plan for the consumer’s delivery state to start over.

If the consumer isn’t meant to be pinned

If the consumer deliberately uses overflow, prioritized or no policy, the metadata is what’s wrong. The same applies while servers are older than 2.11.0 and the policy can’t be set yet. Remove the pinned declaration:

nats consumer info STREAM_NAME CONSUMER_NAME -j | \
  jq '.config | del(.metadata["io.nats.monitor.pinned"])' > consumer.json
nats consumer edit STREAM_NAME CONSUMER_NAME --config consumer.json

Don’t use the --metadata flag for this: it replaces the consumer’s entire metadata map with whatever you pass. You can also remove the key via nats consumer edit STREAM_NAME CONSUMER_NAME --interactive.

Prevent future mismatches

Add validation to your consumer creation workflow. Any process that sets io.nats.monitor.pinned metadata should also set priority_policy=pinned_client and a priority group. This can be enforced via CI checks on IaC templates or wrapper scripts around consumer creation.

Use Synadia Insights for ongoing drift detection. Insights evaluates this check across all consumers every collection epoch, catching mismatches introduced by manual changes, failed deployments, or configuration drift between environments.

Frequently asked questions

How is pinned_client different from overflow and prioritized?

They are three separate priority policies. pinned_client (2.11.0+) gives one client at a time exclusive delivery, with the others waiting to take over. Under overflow (2.11.0+), clients that set min_pending or min_ack_pending on their pull requests receive messages only when the consumer’s pending counts exceed those thresholds — they act as spillover capacity that engages under backlog. prioritized (2.12+) serves lower-numbered priorities first. Only pinned_client satisfies this check.

Can I use pinned clients with push consumers?

No. Priority groups require pull consumers, and the server rejects push consumers that set a priority policy (error 10178, “priority groups can not be used with push consumers”).

How do I move the pin to another client?

Stop the pinned client and wait for priority_timeout to pass, or unpin it explicitly:

nats consumer unpin STREAM_NAME CONSUMER_NAME workers

The server clears the pin and pins the next client with a waiting pull request. A client still sending the old pin ID receives a 423 pin ID mismatch status; the nats.go jetstream package handles this by dropping the stale ID and pulling again.

What happens if I have pinned metadata but no clients?

The consumer exists but nothing is delivered, and no client is pinned. This check stays quiet as long as the policy is pinned_client, because it compares configuration only. The upstream nats server check consumer health check does report this state, as 0 / 1 pinned clients.

Is the pinned metadata required to use the pinned_client policy?

No. You can set priority_policy=pinned_client without the metadata. The metadata is a monitoring declaration, used by Synadia Insights and nats server check consumer to verify that the operator’s intent matches the configuration. Setting the policy without the metadata is valid, but the consumer won’t be covered by this consistency check. Insights also requires io.nats.monitor.enabled=true — io.nats.monitor.pinned on its own is ignored.

What priority groups should I use?

A priority group scopes the client population of a single consumer: clients name the group on their pull requests, and it must be one of that consumer’s configured priority groups. Groups on different consumers are independent by construction, so you don’t need distinct group names across consumers to keep their behavior separate. A single group per consumer is the standard configuration today — the server currently uses only the first configured group.

Does changing priority_policy disrupt active clients?

Yes, by design. Once the policy is pinned_client, pull requests must name a priority group, and only one client receives messages. Deploy clients that pull with the group before or together with the configuration change, and expect the clients that lose out on the pin to go idle until they are needed.