About HarperDB
HarperDB is a distributed systems platform combining an ultra-fast document store, in-memory cache, real-time message broker, and application components. With HarperDB, customers can deliver global-scale backend services with less effort, higher performance, and lower cost.
Overview
HarperDB needed a way to synchronize data across a global network of distributed nodes without increasing complexity or cost. By integrating NATS.io, they achieved massive scalability, real-time replication, and operational simplicity across edge, on-prem, and cloud environments.
Challenges
HarperDB, a leader in distributed systems technology, was facing a challenge in synchronizing data and scaling across geo-distributed nodes. As a result, developers were losing valuable time managing the infrastructure. The company’s goal was to ensure reliable data transactions and communications across a global network of nodes, crucial for maintaining performance and data integrity. HarperDB wanted a highly reliable, scalable and simple-to-manage platform that could handle complex data replication in real time while reducing operational complexity, management and costs.
- Managing data transactions across a global network of nodes
- Supporting limitless global scale without adding complexity/cost
- Reducing data and service latency and increasing system resiliency and reliability
- Ensuring real-time data replication, data integrity and data delivery guarantees
Why NATS?
HarperDB integrated NATS, leveraging its distributed persistence and global scalability. NATS enabled HarperDB to deliver resilience, extensibility and limitless global scaling with minimal complexity. NATS also facilitated real-time, mesh-powered replication across nodes in disconnected environments, ensuring continuous operations even when network connectivity was intermittent. This capability was critical for HarperDB’s topology, which comprises hubs and leaf nodes, enabling seamless functionality across different environments, including autonomous drone networks and cloud platforms.
- Open source, cloud-native, cloud-agnostic messaging & communications platform
- Scalable, low latency & resilient system – no multiple specialized tools or middleware
- Simple to use, lightweight & fast
- Includes NATS JetStream, a highly configurable message persistence layer across nodes
Results
Integration with NATS allowed HarperDB to handle frontend and backend messaging effectively, supporting real-time data replication across nodes and direct-to-client pub/sub communication.
- Productivity: By offloading the message management to NATS, HarperDB’s developers focused on enhancing core database features.
- Reliability: NATS’ exactly-once delivery ensured data replication and eliminated the risk of data loss even at scale.
- Flexibility: With NATS’ bidirectional and unidirectional data replication, HarperDB 4.0 simplified complex use cases like edge machine learning and real-time data processing.
- Scalability: With NATS and its unlimited throughput, HarperDB scaled clusters without limits, unlocking horizontal scaling and saving millions in costs.
- Data Persistence: JetStream for data persistence handled transaction logs and work queues, while maintaining global transparency for audits and troubleshooting.
- HA/Resilience: NATS’ clustering capabilities ensured redundancy and failover capabilities, reducing downtime and data loss.
- Reduced Latency/Global Distribution: NATS’ leaf nodes ensured streams were maintained across edge and on-premises environments – even with limited or intermittent network connectivity.
- Simplified Service Discovery: NATS’ clustering capabilities formed the backbone of HarperDB’s mesh architect
