RethinkConn is back — the biggest NATS event of the year returns June 4. Save your (virtual) spot.
All posts

A community member asked how to translate a familiar RabbitMQ exchange-and-queue routing model into NATS JetStream, especially when different applications need overlapping sets of messages.

The short answer: in JetStream, avoid thinking of each consumer application’s route as a separate physical queue that must duplicate storage. A common pattern is to store related messages once in a stream and create one or more consumers that each deliver the subset an application needs.

The key difference: streams store, consumers deliver

RabbitMQ routing commonly starts with exchanges, bindings, and queues. If the same message should be routed to multiple queues, the broker can fan that message out into multiple queue backlogs.

JetStream has a different model:

  • Subjects are the routing namespace used by NATS.
  • Streams persist messages matching configured subjects.
  • Consumers are views over a stream that track delivery state and can filter which subjects they receive.

A JetStream stream captures a set of subjects. Within the same account, JetStream does not allow two streams to capture overlapping subjects: if you try to create a second stream whose subjects overlap those of an existing stream, the server rejects it. (Mirrors and sourced streams are a separate mechanism — they copy messages from another stream rather than capturing subjects directly, so they are not bound by this rule in the same way.) Because of this, the idiomatic approach is to place related subjects into one stream and create multiple consumers over that stream.

For example:

1
Producer publishes:
2
subject1
3
subject2
4
subject3
5
6
Stream EVENTS stores:
7
subject1
8
subject2
9
subject3
10
11
ConsumerApp1 uses a consumer filtered to:
12
subject1, subject2
13
14
ConsumerApp2 uses a consumer filtered to:
15
subject2, subject3

In this model, subject2 is stored once, but can be delivered independently to both applications through separate consumers.

Do not create one stream per route by default

If you are coming from RabbitMQ, it can be tempting to model every queue or route as a separate stream. That is usually not the best starting point in JetStream.

Creating one stream per individual subject can work, but it has tradeoffs:

  • Applications that need multiple subjects may have to coordinate reads from multiple streams.
  • You lose a single ordered log across those related subjects.
  • Operationally, you may create many more streams than necessary.
  • Cross-subject workflows become harder to reason about.

A more idiomatic JetStream design is often:

1
Stream ORDERS subjects:
2
order.>
3
4
Example published subjects:
5
order.created
6
order.address_updated
7
order.cancelled
8
order.shipped

Then each application creates or uses a consumer filtered to the order events it needs.

JetStream preserves ordering within a stream. If related events are split across multiple streams, there is no single stream order across all of them.

That matters for workflows such as event sourcing, audit trails, or business processes where relative ordering between event types is important. For example, if order.created and order.cancelled are part of the same business timeline, keeping them in the same stream can make consumers easier to reason about.

A filtered consumer over one stream receives the matching messages according to that stream’s order. That is different from merging messages in application code from several independent streams.

A useful rule of thumb is to scope streams around a business area, data domain, or retention requirement, rather than around a single consuming application.

Examples:

  • ORDERS stream for order.>
  • PAYMENTS stream for payment.>
  • INVENTORY stream for inventory.>

This allows many applications to consume from the same durable record of related events without each application needing to own a separate copy of the data.

The boundary is still a design choice. If two sets of subjects need very different storage, retention, replication, or operational handling, that may justify separate streams. If they are part of the same business log and share similar lifecycle requirements, one stream with filtered consumers is often simpler.

What about different retention policies?

A stream has exactly one retention policy. Consumers can differ in delivery state, acknowledgement behavior, filtering, durability, and other consumer-level settings, but they do not give the same underlying messages different stream-level retention policies.

This is an important design constraint. JetStream provides three retention policies:

  • Limits (LimitsPolicy) is the default. Messages are kept until a configured limit is reached — age, total size, or message count — independent of consumers. Any number of consumers can each independently read, and re-read, all messages. This is the natural fit for delivering the same events to multiple applications.
  • Interest (InterestPolicy) keeps a message only while consumers still have interest in it: a message is removed once every consumer that was bound when it was published has acknowledged it. Multiple consumers are allowed, but a consumer must exist before a message is published to receive that message.
  • Work queue (WorkQueuePolicy) is intended for work that should be processed once. A message is removed as soon as it is acknowledged, and the stream only permits consumers with non-overlapping filter subjects — effectively one consumer per subject. If two independent applications each need to receive the same subject, WorkQueuePolicy is not the right fit for that stream.

One clarification on work-queue streams: “one consumer per subject” does not mean a single worker. Many worker instances can pull from the same consumer in parallel to share the load — that is load balancing, not duplication. What a work-queue stream does not support is two separate consumers, and therefore two independent applications, each receiving their own copy of the same subject.

When multiple applications each need their own copy of the same messages, use a limits-based stream (or, with the caveat above, an interest-based stream) and give each application its own consumer. If one application truly needs work-queue semantics while another needs replayable history for the same logical event, you may need to revisit the subject design, stream boundaries, or whether those are actually two different message flows.

A practical design checklist

When deciding how to model routing in JetStream, ask:

  1. Are these subjects part of the same business domain?
    If yes, they are often good candidates for one stream.

  2. Do consumers need a shared order across these subjects?
    If yes, keeping them in one stream is important.

  3. Do the subjects need the same stream-level retention and storage behavior?
    If yes, grouping them is simpler. If no, separate streams may be appropriate.

  4. Is a message a durable event or a unit of work?
    Durable events are often consumed by multiple independent consumers. Units of work that should be processed once may fit a work-queue pattern.

  5. Is the stream owned by a business domain or by a consuming application?
    Prefer avoiding a model where one consumer application’s private setup becomes a shared dependency for unrelated applications.

Who should create streams and consumers?

There is no single required deployment model. NATS gives you options, and the right choice depends on your environment.

Common patterns include:

  • Application-managed streams and consumers: the app creates what it needs at startup.
  • Infrastructure-managed streams: streams are defined through deployment automation, while applications create or bind to their consumers.
  • Fully infrastructure-managed JetStream resources: both streams and durable consumers are provisioned outside the application.
  • Hybrid models: for example, platform teams manage streams while teams manage their own consumers.

A common practical compromise is:

  • Create and manage streams as shared infrastructure or as part of the producing domain.
  • Let consuming applications create or manage their own consumers, especially when those consumers are application-specific.

This avoids making a shared stream accidentally belong to one consumer app. It also makes stream-level changes more deliberate, which matters when multiple producers or consumers depend on the same stream.

That said, app-managed streams can be reasonable when there is a single producer type that clearly owns the stream and the deployment order is controlled. The tradeoff becomes more complex when many app instances or multiple producer types might all try to create or update the same shared stream.

Example: replacing exchange routing with filtered consumers

Suppose you previously had a producer sending messages that RabbitMQ routed into several queues:

1
ProducerApp
2
-> Queue for ConsumerApp1: subject1, subject2
3
-> Queue for ConsumerApp2: subject2, subject3

A JetStream-oriented design could be:

1
ProducerApp publishes to NATS subjects:
2
subject1
3
subject2
4
subject3
5
6
Stream EVENTS stores:
7
subject1
8
subject2
9
subject3
10
11
ConsumerApp1:
12
consumer on EVENTS filtered to subject1 and subject2
13
14
ConsumerApp2:
15
consumer on EVENTS filtered to subject2 and subject3

The important shift is that routing fanout does not require duplicating persisted messages into multiple queues. The stream is the stored log; consumers are independent delivery views over that log.

Practical guidance

When moving RabbitMQ-style routing patterns to NATS JetStream, start by grouping related subjects into streams and using filtered consumers for each application’s view of the data. Create separate streams when you need different stream-level policies, independent ordering domains, or clearly separate business areas. Be especially careful with work-queue retention: it is for work that should be consumed once, not for broadcasting the same message to multiple independent applications.


Want help from the NATS experts? Meet with our architects to get help tailored to your use case and environment.

Related posts

All posts
Get the NATS Newsletter

News and content from across the community


Cancel