Beyond streaminglog processing

NATS JetStream and Apache Kafka overlap in functionality but differ in architecture.

Apache Kafka is a distributed append-only log processing system designed for write throughput and at its core, log persistence and replay is its sole function. However, that core publish/consume functionality is limited which is why the Confluent Platform includes other components — each running as separate JVM processes — to offer a more complete set of functionalities. If you want to do things such as:

• Filtering of messages in topics
• Merging topics into other topics
• Use streams as Key/Value tables or as queues

Then you will have to also use Apache Kafka Streams, Flink, MirrorMaker and others. This means that you need to run many more JVMs than just the Kafka brokers (and Zookeepers if you don't use or haven't migrated to version 4), and that some of the data will be constantly shuffled back and forth between the Kafka broker and Kafka Streams JVMs and may end up being stored multiple times over multiple topics, affecting latency and efficiency, resource usage and, when using cloud services, can add noticeably to the costs (networking, storage).

NATS consolidates messaging, streaming, Key/Value, Object Store, queue semantics, service discovery, multi-region super-clustering, edge leaf nodes, MQTT, and WebSockets into a single binary with a single operational surface (less than 20MB binary) with zero dependencies. The practical consequences of this architectural difference show up across every dimension compared in this paper.

Kafka 4 has closed some historical gaps but the architectural differences, however, remain: Kafka's model is a partitioned append-only log extended by external components, while NATS JetStream is a consistent addressable message store with messaging, queuing, and data storage built in.

Neither system is universally better. Kafka's ecosystem depth, transactional model, and throughput optimizations make it the stronger choice for large-scale log processing with deep Flink/Iceberg/Schema Registry integration. NATS JetStream's unified architecture, operational simplicity, compliance-friendly deletion, multi-tenant security, and edge deployment capabilities make it the stronger choice for teams building modern distributed applications that span messaging, streaming, and data storage in a single platform.

The two are not mutually exclusive — they bridge cleanly via Synadia Connect, Debezium Server, or direct protocol adapters, and can coexist cooperatively in the same architecture.

This paper has argued that NATS with JetStream persistence is not merely a lighter-weight Kafka alternative but a fundamentally different architecture — one that collapses the distinction between messaging, streaming, queuing, and data storage into a single system with a single binary and a single operational surface. The differences that matter most fall into five categories.

Architecture: Kafka's model separates concerns across multiple process types: brokers, controllers, Connect workers, Schema Registry, stream processing runtimes (Kafka Streams, Flink), and replication tools (MirrorMaker, Cluster Linking). Each adds operational surface. NATS consolidates messaging, streaming, Key/Value, Object Store, queue semantics, multi-region topology (super-clusters), edge connectivity (leaf nodes), WebSockets, MQTT, service discovery, and stream mirroring into a single nats-server binary. Features are enabled in configuration, not by deploying new infrastructure.

Addressing and storage: Kafka stores data in partitioned, append-only topic logs addressed by offset. NATS JetStream stores data in streams that are addressed by hierarchical subjects with wildcards, support individual message deletion, can enforce per-subject constraints, and expose atomic compare-and-set write control. This makes JetStream simultaneously a stream, a key/value store, an object store, and a durable work queue — without external dependencies. It also makes targeted data deletion for GDPR and other compliance requirements a single API call rather than an architectural workaround.

Consumption: Kafka ties consumer parallelism to partition count and — even with share groups — retains partitions as a foundational abstraction. JetStream consumers are partition-less by default: scaling consumers up or down is a client-side operation with no server-side rebalance. For workloads that genuinely need subject-based partitioning, the Orbit client library extensions provide it as an opt-in layer, not a mandatory constraint.

Multi-tenancy, security, and deployment: NATS provides account-level multi-tenancy with isolated subject namespaces, delegated administration via JWTs and scoped signing keys, and a security callout mechanism for integrating with external identity providers. Deployment topologies span single clusters, super-clusters across regions and cloud providers, and leaf nodes at the edge with store-and-forward resilience during connectivity gaps. Kafka has no native equivalent to accounts, super-clusters, or leaf nodes.

Observability: NATS embeds monitoring into the server process: JSON telemetry endpoints, a push-based system account event channel, and — since 2.11 — built-in distributed message tracing with a single CLI command. Kafka's observability depends on JMX, external exporters, and client-side OpenTelemetry instrumentation, with no server-side message tracing capability.