Pub/sub messages

Messages are pieces of information, such as business events taking place in an integrated environment, that endpoints exchange in the publish/subscribe architecture.

Each message may represent a single occurrence of a specific change of state - for instance, an update to a user's phone number which is information that may need to be delivered to multiple systems and services that keep track of the user's current state. On the other hand, an IoT system may want to publish a steady stream of events, e.g. continually broadcasting its current state to interested subscribers. Yet another example is a request-response scenario where the response is expected to arrive after a relatively longer time, e.g. after minutes, hours or days, which implies that the requesting application should not wait for the response in a blocking manner.

All messages exchanged through Zato publish/subscribe are always wrapped in JSON or Python dicts - this conveniently allows any kind of an application to send or receive them, regardless of whether they are implemented in Python, Java, JavaScript or any other technology.

Because all REST pub/sub message use JSON, it is possible to use standard and common tools, such as curl or Postman, during the development or testing.

Data of any type can be sent embedded in JSON, e.g. it is possible to send CSV other types of information. There are no message size limits although, typically, each message will not be bigger than a few dozen kilobytes.

Each message is composed of the actual business data as well as metadata - a set of attributes that clarify, refine and define additional routing or delivery options. No matter if it is REST, Python or other endpoint sending or receiving messages, all message attributes are always consistently named the same, e.g. a subscription key is always "sub_key" and the actual data is always in an attribute called "data".

The following is a discussion of each of the attributes that describe each pub/sub message. Some of the attributes are writable - they can be provided on input, during publication, whereas some are populated by Zato and they become only available to recipients. If an attribute is optional, it means that either such a key may not exist or it may contain an empty value.

All the attributes are discussed using the following sample message that has been already received by a subscriber.

{
 "data":"This is a sample message",
 "topic_name":"/zato/demo/sample",
 "msg_id":"zpsm3a91d3e76a325b60700c1c66",
 "correl_id":"abcdef123456",
 "in_reply_to":"",
 "priority":5,
 "mime_type":"application/json",
 "ext_client_id":"MyApp.1",
 "pub_time_iso":"2023-04-03T15:59:04.760628",
 "ext_pub_time_iso":"",
 "recv_time_iso":"2023-04-03T15:59:04.941140",
 "expiration":86400,
 "expiration_time_iso":"2023-04-04T15:59:04.760628",
 "size":24,
 "delivery_count":7,
 "sub_key":"zpsk.rest.4fed61",
}

Name	Required	Writable	Notes
data	---	Yes	The data that was published to the topic from topic_name.
topic_name	Yes	Yes	Name of the topic that the message was published to.
msg_id	Yes	---	A unique message ID - all IDs start with the prefix of "zpsm", which stands for "Zato pub/sub message".
correl_id	---	Yes	An optional correlation ID provided by the publisher. This can be any string that lets endpoints correlate messages in a series, e.g. it can be an invoice number or any other business or non-business identifier. It does not have to be unique.
in_reply_to	---	Yes	ID of the message that this message is a reply to.
priority	Yes	Yes	Each message has a priority from 1 to 9 (max=9). The higher the priority the higher in a given subscriber's delivery queue the message will be stored and it will be delivered before messages of lower priority. Default priority is 5.
mime_type	Yes	Yes	MIME type that the data is in. This is for the benefit of subscribers only, Zato does not use it.
ext_client_id	---	Yes	An arbitrary ID of the endpoint that publishes the message. It may be, for instance, name of a server instance, its IP, or perhaps a serial number or a MAC address of an IoT device that publishes the message - it is up to the publisher to decided if any such ID should be used.
pub_time_iso	Yes	---	When the message was published, in UTC, as it was observed by Zato, using its local clock.
ext_pub_time_iso	Yes	Yes	When the message was published, in UTC, as it was observed by the publisher, using the publisher's clock. This may be provided on input by the publisher and subscribers may prefer to consult it over pub_time_iso in case there are any discrepancies between the two, e.g. if clocks go out of sync.
recv_time_iso	Yes	---	When Zato received internally the message for delivery, in UTC. This will be always greater than pub_time_iso.
expiration	Yes	Yes	After how many seconds to consider the message as expired and stop any attempts at its delivery. Default is 86400 seconds = 1 day.
expiration_time_iso	Yes	---	Expiration time as a timestamp, the value is recv_time_iso + expiration.
size	Yes	---	The size of the data attribute - not the entire message - in bytes.
delivery_count	Yes	---	How many times Zato attempted to deliver this message to the subscriber receiving it. If it is equal to 1, it means that it is the first attempt. If it is greater than 1, it means that there previous attempts that failed, e.g. the subscriber was not available for some time.
sub_key	Yes	---	The subscription key of the subscriber receiving the message.