About Intelecy
Intelecy is a No-Code AI platform that helps industrial companies optimize factory and plant processes with real-time machine learning insights. These insights allow process engineers to manually or automatically improve outcomes, quality and sustainability without requiring significant custom code or hardware. Designed to help factories move towards greater automation and true “lights out” operational capabilities, Intelecy securely collects data from tens of thousands of sensors in a factory and pushes that data into the cloud using NATS and the Synadia Cloud global data fabric.
Overview
The Intelecy platform runs data through machine learning models customized for each industrial process in a plant and returns analysis and insights back to the sensors and controlling systems in near real time, with less than one-second round-trip latencies.
Intelecy can ingest and process data in all widely used industrial formats including SCADA, HMI, DCS, and MES. Intelecy customers come from a variety of industrial sectors including food and beverage, materials, power generation and mining. Intelecy runs on a cloud native architecture with loosely coupled services utilizing containers orchestrated by Kubernetes/Nomad, the Envoy proxy, gRPC for messaging, ClickHouse for time series data storage and NATS for data streaming and intelligent persistence.
Challenges
Intelecy is operating in a sector that changes slowly due to the risks of production interruption; shutting down a factory or a production line, even for a few minutes, is costly. The industrial sector also operates using a wide array of systems and sensors, requiring flexible integrations and ability to interact with and control several types of different operational technology (OT) assets and protocols.
As a design convention, Intelecy wanted to leverage managed services running in the cloud as much as possible for non-core functionality. The engineering team also realized quickly that they would need to make modifications on the original architecture to reduce latency of data insight delivery and improve segmentation to handle multi-tenancy. In addition, they wanted to update the design of their application and services stack to be as cloud agnostic and container-centric as possible, avoiding inclusion of cloud-specific services that could create a lock-in or a bottleneck.
Intelecy initially designed its platform using poll-based systems that were tightly tied to Microsoft Azure services. Those services included SQL data stores but not time series, which was a requirement for many industrial customers. In addition, the poll-based architecture could return processed data and insights back to customers with a delay of 15 to 30 minutes.
- Low Latency: Less than 2 seconds round-trip time for processing streamed data and returning ML-driven insights to customers
- Segmentation and Security: Secure and simple customer segmentation of data streaming in a multi-tenant environment
- Portability: A cloud-native and edge-friendly solution that was portable and cloud agnostic
- Flexibility: Ability to restructure client topology without restarts, and handle varying message sizes
- Scalability: Scale data streaming footprint up and down quickly, without restarts or major configuration changes
- Simple Security: Security as a core element, not an add-on layer requiring additional overhead
Why NATS
Intelecy had worked with NATS before and immediately considered it as the data streaming solution for their application requirements. Chief Security Officer Jonathann Camp liked the simplicity of NATS and was interested in the NATS utility SaaS offering, Synadia Cloud. Synadia Cloud would eliminate the need for Intelecy to maintain a data streaming infrastructure and only consume as much capacity and bandwidth as it needed.
Camp also liked the manner in which Synadia’s team (the core maintainers of NATS) worked on the software in public and interacted with issues, PRs and community members. “The way someone develops software tells you a lot about its quality. And the fact that the creators of NATS are so heavily involved in the code on a day-to-day basis gave me confidence,” says Camp.
Intelecy did look at a handful of other streaming and messaging applications, including some using MQTT, but found them to be less flexible and requiring more resources for stand up and maintenance. Intelecy briefly looked at Apache Kafka but decided against using it extensively for data streaming. “Kafka has requirements that it puts on the client,” says Camp. “The client has to be aware of a lot of things. I never liked that design paradigm. I wanted the client to be really simple.” The worker queues model, as well, exposed complexity and made changing the topology of the application challenging.
After a quick decision process, Intelecy opted to replace its poll-based pub-sub data fabric with Synadia Cloud. “The hosted service closed the deal,” says Camp. “We are a small startup. We don’t have time to manage everything.”
- Open source, cloud-native, and hosted messaging & communications platform
- Simple client design without complex topology or worker queue requirements
- Flexible and lightweight for easy scaling and topology changes without restarts
- JetStream integration with granular control over durability and persistence
- Security by default with least-privilege model built into JSON Web Tokens
- Cloud agnostic allowing movement between cloud environments in minutes
Technical Benefits
- Simple extensible method for commanding devices behind firewalls
- Handles streaming of data of various message sizes
- Gracefully handles intermittent connectivity
- Global control plane using RPC and protobufs
- Performant with minimal infrastructure management overhead
- Multi-tenant security simplified through subject-based segregation
Results
With Synadia Cloud and JetStream, Intelecy successfully built a super-fast bi-directional data pipeline that enables real-time machine learning insights for industrial customers.
- 15-30x reduction in roundtrip latency: From 15-30 minutes down to less than 1 second
- Cloud and infrastructure agnostic: No lock-in, move between clouds in minutes
- Requires zero maintenance: Managed service eliminates operational overhead
- Improved resiliency and higher availability: Leaf Nodes enable high availability configurations
- Improved security for multi-tenancy: Least-privilege by default security model simplifies customer segmentation
- Eliminated need for complex proxy or service mesh: Built-in security and routing
- Enables combination of data streaming and batch: JetStream provides persistence and durability
- Handles large message sizes: Flexible message handling for various industrial data formats
- Onboard and manage customers easily: Simple scaling by adding/removing workers without configuration changes
- Less technology complexity: Simplified architecture with fewer moving parts
- Reduced operational overhead: Managed service reduces infrastructure management burden
- Easy scalability: Add or remove workers dynamically without restarts
