Streaming Costs Comparison: NATS vs. Kinesis Compute
When architecting data streaming solutions, throughput, latency, and reliability are all key considerations. But what about efficiency and the bottom line?
Above, a table from the paper with the NATS total costs used as comparisons to Kinesis throughout the analysis. Get the full Kinesis TCO analysis here.
The compute costs associated with your streaming platform can significantly impact operational expenses, especially at scale. A recent benchmarking analysis by Jean-Noël Moyne, our Field CTO at Synadia, takes a deep look at the cost differences between NATS.io and Amazon Kinesis. Let’s break down a portion of the findings.
The paper defines “compute costs” as the EC2 instance costs for NATS and the “consumer costs” in Kinesis. For NATS, a benchmark 3-node cluster using EC2 c5n.xlarge instances (deployed across three AWS Availability Zones) incurred a straightforward compute cost of $473.04 per month. This figure represents the expense for the servers powering the NATS streaming service.
Understanding Provisioned Kinesis Costs
Of course Kinesis, as an AWS service, presents a more complex picture for its compute expenditure. Its “consumer costs” are not just a simple instance fee. For Provisioned Kinesis, these costs primarily encompass shard hours (the capacity units for your stream) and, crucially, costs associated with Enhanced Fan-Out (EFO) consumers.
EFO is often necessary if you need low latency or have multiple applications consuming the same stream in real-time – functionalities that NATS provides by default. EFO consumer costs include “EFO consumer-shard hours” and “EFO data retrieval costs.”
Understanding On-Demand Kinesis Costs
For On-Demand Kinesis, compute costs are made up of on-demand stream hours (for the stream’s base capacity), on-demand EFO consumer-stream hours, and the per-GB cost of data retrieved via EFO.
Note: “Put Payload Units” (for writing data to Kinesis) are considered storage costs by the paper and are excluded from this compute cost analysis.
The EFO Factor and Its Impact on Kinesis Compute Costs
In the paper, Jean-Noël shows how NATS’s out-of-the-box or default consumer model is functionally equivalent to Kinesis’s more premium EFO consumers. This is where the compute cost disparity becomes very obvious. When EFO is required for Kinesis to match NATS’s capabilities for multiple, low-latency consumers, Kinesis compute costs spike significantly.
Let’s compare the NATS cluster’s fixed $473.04 monthly compute cost with Kinesis’s compute costs (derived from the paper’s figures by isolating “consumer costs”) for similar workloads:
Provisioned Kinesis - Estimated Monthly Compute Costs (Consumer Costs)
| Scenario | 1 EFO Consumer | 4 EFO Consumers |
|---|---|---|
| 1 KiB messages @ 45 MiB/s | $2,561.16 | $8,602.12 |
| 10 KiB messages @ 125 MiB/s | $7,029.17 | $23,681.92 |
| NATS Compute Cost (Fixed) | $473.04 | $473.04 |
As the number of EFO consumers increases, or as throughput demands necessitate more capacity (translating to higher shard and EFO-related costs), Kinesis’s compute expenses rise sharply compared to the stable NATS compute cost.
On-Demand Kinesis - Estimated Monthly Compute Costs (Consumer Costs)
| Scenario | 1 EFO Consumer | 4 EFO Consumers |
|---|---|---|
| 1 KiB messages @ 45 MiB/s | $14,960.80 | $31,608.02 |
| 10 KiB messages @ 125 MiB/s | $40,755.86 | $87,748.16 |
| NATS Compute Cost (Fixed) | $473.04 | $473.04 |
The On-Demand Kinesis model, while offering scaling convenience, has even higher compute costs than Provisioned Kinesis. Is the convenience of Kinesis, On-Demand or Provisioned, worth 5-20x the cost of running a NATS cluster?
What’s Next?
Focusing only on compute costs, the NATS architecture, with its predictable EC2 instance-based pricing for the streaming cluster, has a very different cost profile than Amazon Kinesis. For scenarios requiring multiple consuming applications or low-latency data delivery – where Kinesis users would typically need to employ EFO consumers – the compute costs for Kinesis can become orders of magnitude larger than those for a comparably performant NATS cluster.
Evaluating Kinesis for streaming data needs in your architecture? Before you make an expensive decision, check out the complete Kinesis TCO analysis here.
Up next in this series: a review of storage costs in the NATS vs Kinesis TCO comparison.
