Connector Series Recap: Building Real Data Pipelines with NATS
Your NATS data plane is moving messages at scale. Now you need to integrate with external systems—push events to S3 for compliance, sync state to MongoDB, trigger webhooks when conditions match. Traditionally, this means writing custom integration code, managing error handling, and maintaining bespoke connectors for each destination.
NATS Connectors eliminate this friction. This three-part video series demonstrates how to build complete data pipelines—moving data out to external systems (Outlets), pulling it back in (Inlets), and reshaping it along the way (Transformers)—all without writing custom integration code. You’ll see the complete picture: how to push messages out to external systems, pull data back in from those same systems, and reshape everything so one stream can serve multiple destinations. Three episodes, one full loop you can follow from start to finish.
Why this three-part structure?
Most data pipeline tutorials focus on either ingestion or delivery, but rarely show the complete loop. We started with Outlets for a reason: before you can demonstrate ingestion, there has to be something real to ingest. So we took messages flowing through NATS and pushed them into places everyone recognizes—S3 buckets, MongoDB collections, and live webhook endpoints. Objects you can browse, documents you can query, HTTP calls you can trace in real time.
With those systems populated, Inlets pulled that same data back into NATS, closing the loop with a clean round trip. Finally, Transformers reshaped everything in flight—one stream, multiple formats, no changes to producers. By the end, you’ve seen the whole arc: make it visible, bring it home, mold it to fit.
Episode 1: Outlets (seeding reality)
Outlets subscribe to subjects and deliver payloads to external systems. Starting here lets us create visible proof that the pipeline works and set up real sources for the next step. You’ll literally see files land in a bucket, new docs appear in a collection, and a webhook receiver log light up as messages move.
What we built:
- Amazon S3 outlet writing messages as objects (you’ll see a clear prefix/pattern)
- MongoDB outlet upserting records into a collection you can immediately query
- Webhook outlet (ngrok) calling a live HTTP receiver with the payload you just published
What to watch for: A quick publish on a NATS subject → visible artifacts in S3/Mongo/webhook logs within seconds.
Episode 2: Inlets (closing the loop)
With data now parked in external systems, Inlets became the natural follow-up. We pointed them at the destinations we had just populated and streamed that reality back into NATS. The result is a simple round trip you can see end-to-end: messages originate in NATS, land in external systems, and return on demand for further routing or processing. It’s the same data, just entering from the outside this time, which makes the flow feel concrete rather than theoretical.
What we built:
- Amazon S3 inlet reading the objects created in Episode 1 (plus any existing files)
- MongoDB inlet consuming database change streams in near real time
- Webhook inlet (ngrok) receiving events from third parties without extra scaffolding
What to watch for: Inlets emit onto NATS subjects you can subscribe to immediately—publish/subscribe windows side-by-side make the loop obvious.
Episode 3: Transformers (shaping the middle)
Watch the Transformers episode →
After data could move in both directions, we added transformation logic in the middle. Transformers let you change structure and meaning without touching producers or destinations. That’s where pipelines become adaptable—one stream can serve analytics, ops, and automation with different needs and different shapes, all from the same flow. You’ll see how a small tweak in the transformer yields exactly the format a destination expects.
What we built:
- Mapping transformer to rename, restructure, and tidy fields with a quick, readable config
- External Python service transformer to enrich/modify payloads via a lightweight service you control
What to watch for: A before/after payload view that makes the transformation instantly clear.
The complete picture
By combining Outlets, Inlets, and Transformers, you can build complete data pipelines that integrate NATS with your existing infrastructure—without writing custom integration code or managing complex ETL tooling. NATS Connectors handle common operational concerns like retries, dead letter queues, batching, and error handling, so you can focus on your business logic instead of integration plumbing.
Watch the series in order:
- Outlets → S3, MongoDB, webhook deliveries you can see right away
- Inlets → ingest from those same systems back into NATS with a couple configs
- Transformers → restructure with mapping plus a small Python service (quick before/after demo)
Next steps:
- Watch the full series to see how the components connect
- Explore the Synadia Connectors documentation for additional destinations and transformation options
- Try building your own pipeline with the connectors available in Synadia Platform and Synadia Cloud
Start building data pipelines today
NATS Connectors are available in both Synadia Platform for on-premises and private cloud deployments, and Synadia Cloud for teams getting started with fully managed NATS infrastructure.
With Synadia Cloud, you get built-in monitoring and management through our Control Plane UI, so you can focus on building pipelines instead of managing servers. NATS clients are available in every major language, making it simple to integrate with your existing applications.
Start with Synadia Cloud for free or explore Synadia Platform for production deployments.
