What is publish/subscribe?

Publish/subscribe (pub/sub) is a messaging pattern in which the systems that produce information don't send it directly to the systems that consume it. Instead, producers send messages to an intermediary - a message broker - and it is the broker that delivers the messages to consumers. Any system that can make an HTTP call can participate - regardless of programming language, operating system or deployment model.

This one idea has far-reaching consequences for how you design, build and operate integrations.

The core concept

Consider an employee onboarding system. When a new hire's paperwork is finalized, several systems need to react: the payroll system, the benefits enrollment portal, the training platform and the access provisioning service.

Without pub/sub, the onboarding system would need to know about each consumer and call each one directly. Every time you add a new consumer, you modify the onboarding system. Every time a consumer goes offline, the onboarding system's calls fail.

With pub/sub, the onboarding system publishes a message to a topic - say, employee.onboarded - and the broker delivers it to all systems that have subscribed to that topic. The onboarding system doesn't know or care who the subscribers are. Subscribers don't know or care who the publisher is.

What does the broker do?

The broker is the intermediary that makes this work. It sits between publishers and subscribers and takes care of:

• Routing - delivering each published message to the queues of all matching subscribers
• Queue management - maintaining a personal queue per subscriber so each one works at its own pace
• Security - controlling who can publish to which topics and who can subscribe to them
• Runtime flexibility - letting you add or remove subscribers at any time without touching publishers or restarting anything

Publishers

With the broker's role clear, let's look at each side of the conversation. A publisher is any system that sends messages to a topic. It only needs to know two things: the name of the topic and the format of the message. It doesn't need to know who will receive the message, how many subscribers there are, or whether any subscribers exist at all.

In Zato, publishers can be REST API clients or Python services. A REST client authenticates with HTTP Basic Auth and posts a message to a topic endpoint. A Python service calls self.publish and provides the topic name and data.

Topics

Publishers don't send messages to subscribers - they send them to topics. A topic is a named channel to which messages are published. Think of it as a mailbox with a label - publishers drop messages into it, and the broker takes care of routing those messages to the right recipients.

Topics in Zato support wildcard patterns. A topic named orders.* matches orders.created, orders.shipped, orders.cancelled. A topic named invoice.** matches invoice.emea.received, invoice.apac.approved.urgent, and any other path that starts with invoice. regardless of depth.

Subscribers

On the other end of a topic are subscribers. A subscriber is any system that has declared interest in receiving messages from one or more topics. Once subscribed, it receives every message published to those topics from that point forward.

There are two ways a subscriber can get its messages:

• Pull - the subscriber connects to the broker when it's ready and retrieves waiting messages. This gives the subscriber full control over timing and pace.

• Push - the broker delivers messages to the subscriber's endpoint as soon as they arrive. This is useful when you need near-real-time processing without polling.

Message queues

What happens to messages between the moment they're published and the moment a subscriber picks them up? When a subscriber subscribes to a topic, the broker creates a personal message queue for that subscriber. Every message published to the subscribed topics is placed into this queue and stays there until the subscriber retrieves it.

This is what makes pub/sub resilient. If a subscriber goes offline for maintenance, its queue accumulates messages. When it comes back, it processes the backlog. Nothing is lost.

Each subscriber has its own independent queue. A slow subscriber doesn't block a fast one. A crashed subscriber doesn't affect anyone else.

Pub/sub vs. point-to-point

Now that the building blocks are in place, it's worth contrasting pub/sub with the alternative. In point-to-point messaging, a sender targets a specific recipient - the sender must know who the receiver is and how to reach it. Pub/sub removes that constraint entirely. A publisher sends a message to a topic and has no knowledge of who will receive it, how many subscribers there are, or whether any exist at all. The broker handles the rest.

Why use pub/sub instead of direct calls?

The difference becomes concrete when you look at a real integration scenario. Consider a company where the order management system needs to notify the warehouse, the billing system, the analytics platform and the customer notification service every time an order is placed.

With direct calls, the order system makes four API calls in sequence. If billing is slow, the warehouse waits. If notifications are down, the entire operation fails or you need complex retry logic for each individual connection.

The problem compounds as you grow. With 5 systems calling each other directly, you manage 20 connections. With 10 systems, it's 90. With a broker in the middle, each system has exactly one connection - to the broker - regardless of how many others participate.

With pub/sub, the order system publishes one message to orders.created. Each consumer processes it independently, at its own pace, on its own schedule. The order system's job is done the moment the broker accepts the message.

The practical benefits:

• Decoupling - publishers and subscribers evolve independently. You can replace, upgrade or scale any system without touching the others.

• Reliability - messages are held in queues until subscribers retrieve them, so temporary network blips or maintenance windows don't cause data loss.

• Scalability - adding a new consumer is a subscription, not a code change in the producer. Going from 2 consumers to 20 requires zero modifications to the publishing system.

• Temporal independence - publishers and subscribers don't need to be online at the same time. A batch system that runs at 3 AM can publish results that a dashboard consumes at 9 AM.

Practical examples

Here's how these concepts come together in two common business workflows.

Employee onboarding

An onboarding system coordinates everything that must happen when a new employee joins. Each step involves a different system owned by a different team.

1. HR finalizes the hire and the onboarding system publishes to employee.onboarded
2. The payroll system subscribes and sets up salary payments
3. The benefits portal subscribes and opens an enrollment window
4. The training platform subscribes and assigns mandatory courses
5. The access provisioning service subscribes and creates accounts in internal tools

When the same employee leaves months later, the onboarding system publishes to employee.departed. The same subscribers react - this time closing payroll, terminating benefits, revoking access. The onboarding system publishes once per event, and doesn't need to track which systems have acted on it.

If the training platform is undergoing maintenance when someone is onboarded, its queue holds the message. The courses are assigned as soon as the platform comes back - no manual intervention, no missed employees.

Invoice processing

A procurement system handles purchase orders and invoices for a mid-size company. When a supplier submits an invoice, multiple departments need to act on it.

1. The procurement system validates the invoice and publishes to invoice.received
2. The accounting system subscribes and records the payable
3. The budget tracking system subscribes and updates remaining departmental budgets
4. The approval workflow subscribes and routes the invoice to the right manager based on amount and category
5. The supplier portal subscribes and updates the invoice status visible to the supplier

Each system processes the event at its own pace. The accounting system may take milliseconds, the approval workflow may take days (waiting for a human). Neither blocks the other.

If the budget tracking system is offline during a deployment, its queue accumulates invoices. When it comes back, it processes the backlog and budgets are up to date again - no invoices are missed, no manual reconciliation is needed.

How Zato implements pub/sub

All of the above is built into Zato. Zato is a message broker that provides pub/sub as a built-in capability. Publishers and subscribers connect using REST APIs or Python services. Security is handled through HTTP Basic Auth with pattern-based permissions that control who can publish to which topics and who can subscribe.

The next pages in this section cover topics and message queues in detail, publishers and subscribers, the REST and Python APIs for working with messages, and security configuration.

Getting started with EDA

Using EDA