Pub/sub - API - REST

Prerequisite reading: Pub/sub architecture.

Overview

Before making use of the REST pub/sub API, a REST endpoint needs to be created. Alternatively, you can get started with the functionality by using the built-in demo account, described at the end of this chapter.

REST integration with publish/subscribe is built around four cornerstone operations:

Operation REST
Publishing POST /zato/pubsub/topic/{topic_name}
Getting messages from queues PATCH /zato/pubsub/topic/{topic_name}
Subscribing POST /zato/pubsub/subscribe/topic/{topic_name}
Unsubscribing DELETE /zato/pubsub/subscribe/topic/{topic_name}

REST calls that provide expected HTTP Basic Auth credentials are able to publish messages to topics, assuming that the caller has proper publication permissions to the topics the messages are sent to.

Each pub/sub endpoint is able to create a subscription provided that the caller uses correct HTTP Basic Auth credentials and that sufficient access permissions are in place to let the endpoint subscribe to a topic of choice. A subscription key is created internally by servers. One endpoint cannot subscribe to the same topic more than once.

At any time, already subscribed endpoints may get newest messages waiting in their queues - this is an operation that changes the server-side state which is why PATCH is used instead of GET.

Applications creating subscriptions using this API are expected to periodically get their messages (pull-style) - as an alternative, if REST subscriptions are created by Zato administrators using web-admin, Zato itself will be able to deliver new messages to the recipients (notify-style).

If a subscription is no longer needed, a REST endpoint may unsubscribe, which will invalidate and delete the endpoint's subscription key.

Publishing messages

Topic name must be provided in URL path. Other parameters are given as JSON payload.

If there are no subscribers to the topic when the message is being published, the message will be retained in the topic. When subscribers arrive, the message will be moved to their queues in background as soon as they create their subscriptions.

MIME type is read from the input Content-Type HTTP header and is turned into text/json if it is equal to application/x-www-form-urlencoded. All the other parameters are sent in JSON requests.

Verb URL
POST /zato/pubsub/topic/{topic_name}

Request

Parameter Datatype Required Notes
data any Yes Actual data of the message that is being published. Can be of any type, e.g. a string or an embedded JSON object.
priority integer --- Message priority from 1 to 9 (1=min). Defaults to 5 if not given.
expiration integer --- Expiration in seconds, there is no default value
correl_id string --- Correlation ID. If the message belongs to a series of messages, this can be used to correlate them.
in_reply_to string --- If the message is in reply to a previous one, this is the field with the value of the previous message's ID
ext_client_id string --- An arbitrary string uniquely identifying the calling application or its instance - used for logging and audit purposes
has_gd bool --- Whether the message that is published should be governed by Guaranteed Delivery (if True) or not. Defaults to False.

Response

Parameter Datatype Required Notes
msg_id string Yes Unique ID assigned to the input message

Samples

OK, data sent and message ID returned:

$ curl -XPOST http://user:pass@localhost:17010/zato/pubsub/topic//zato/demo/sample \
  -d '{"data":"hello", "priority":2}'
{"msg_id": "zpsm1a150dbfb8ab3cb676a471b5"}
$

Error, no data sent on input:

$ curl -XPOST http://user:pass@localhost:17010/zato/pubsub/topic//zato/demo/sample
{
  "result":"Error",
  "cid":"aea73f42fb12382b278e9a3a",
  "details":"Invalid input"}
$

Getting messages from queues

Topic name is provided in the URL path. If a subscription for that topic exists, all messages are returned.

Verb URL
PATCH /zato/pubsub/topic/{topic_name}

Samples

OK, messages returned. Note that messages are always sorted in the Last-In-First-Out (LIFO) order, i.e. from most recently published to the oldest ones. Note also that the current subscription key, the one returned when the client subscribed, is also produced among other details of each message.

$ curl -XPATCH http://user:pass@localhost:17010/zato/pubsub/topic//zato/demo/sample
  {
      "data": "This is a sample message #2",
      "delivery_count": 0,
      "expiration": 998877,
      "expiration_time_iso": "2022-07-04T17:32:46.224820",
      "ext_client_id": "CLIENT-EXT-2",
      "has_gd": true,
      "mime_type": "text/plain",
      "priority": 5,
      "pub_time_iso": "2022-07-04T17:16:07.347820",
      "size": 27,
      "sub_key": "zpsk.rest.ca023c8",
      "topic_name": "/zato/demo/sample"
  },
  {
      "data": "This is a sample message #1",
      "delivery_count": 0,
      "expiration": 998877,
      "expiration_time_iso": "2022-07-17:31:31.043810",
      "ext_client_id": "CLIENT-EXT-1",
      "has_gd": true,
      "mime_type": "text/plain",
      "priority": 5,
      "pub_time_iso": "2022-07-04T17:15:49.809810",
      "size": 27,
      "sub_key": "zpsk.rest.ca023c8",
      "topic_name": "/zato/demo/sample"
  },
$

OK, no messages available:

$ curl -XPOST http://user:pass@localhost:17010/zato/pubsub/topic//zato/demo/sample
[]
$

Error, no such subscription. In this case, the caller still receives an empty list on response ..

$ curl -XPOST http://user:pass@localhost:17010/zato/pubsub/topic//zato/demo/sample
[]
$

.. but at the same time, an entry will be written to server.log indicating that a subscription for that subscriber does not exit in the cluster that the caller is accessing.

INFO - Could not find sub by input `{'cluster_id': 1, 'sub_key': 'invalid'}`

Subscribing

Topic name must be provided in the URL, there are no other request parameters in JSON.

On response, a populated JSON dictionary, described below, is returned if the call succeeded and error details otherwise.

It is not an error to call this endpoint even if the client is already subscribed. If it is, instead of the subscription key and current queue depth, an empty JSON message will be returned and an informational message will be written to server.log, stating that such a subscription already exists, but another one will not be created.

Verb URL
POST /zato/pubsub/subscribe/topic/{topic_name}

Request

n/a

Response

Parameter Datatype Required Notes
sub_key string --- A unique subscription key generated for this client. Must be treated as a secret and guarded accordingly
queue_depth integer --- How many messages are already known to have been enqueued for the calling client in this topic

Samples

OK, subscription created and its details are returned:

$ curl -XPOST http://user:pass@localhost:17010/zato/pubsub/subscribe/topic//zato/demo/sample
{"sub_key": "zpsk.rest.be946f", "queue_depth": 35}
$

Subscription to input topic already exists for this client and its key is returned. Note that the current depth is not returned.

$ curl -XPOST http://user:pass@localhost:17010/zato/pubsub/subscribe/topic//zato/demo/sample
{"sub_key": "zpsk.rest.be946f"}
$

Unsubscribing

Topic name must be provided in the URL path. If the call succeeds, the caller's subscription key will be deleted and the client will have to re-subscribe before messages it will be able to get messages again.

It is not an error to call this endpoint even if the client is not subscribed. However, if it is not, an informational message will be written to server.log, stating that such a subscription does not exist.

Verb URL
DELETE /zato/pubsub/subscribe/topic/{topic_name}

Samples

OK, unsubscribed successfully:

$ curl -XDELETE http://user:pass@localhost:17010/zato/pubsub/subscribe/topic//zato/demo/sample
{}
$

Error - invalid credentials (details are in server.log):

$ curl -XDELETE http://pubsub:abc@localhost:17010/zato/pubsub/subscribe/topic//zato/demo/sample
{}
$

Data format

  • All requests and responses are always in JSON
  • Authentication is always with HTTP Basic Auth
  • On response, either business data is returned if the call was successful or basic error information is returned while all the error details are in server.log

  • Everything is OK:

    {"sub_key": "zpsk.rest.be946f", "queue_depth": 35}
    
  • An error has occurred:

    {
     "result":"Error",
     "cid":"ca3898ee8cbffeaff4086826",
     "details":"You are not allowed to access this resource"
    }
    

Logging

  • All error messages are accompanied by a CID in the JSON response - this is a Correlation ID uniquely assigned to a particular HTTP request
  • CIDs are also always returned in X-Zato-CID response header, regardless if there was an error or not
  • CIDs can be found in http_access.log files of the server that handled a particular invocation
  • In case of an error, file server.log of the server where it happened, contains details, including a full traceback

Demo data

  • All Zato clusters come with a set of demo objects:

  • Topic /zato/demo/sample

  • Endpoint zato.pubsub.demo.endpoint with permissions for publications and subscriptions to topics matching pattern /zato/demo/*

  • The default endpoint has no preset password - it is randomly generated (UUID4) in each environment

  • To make use of the endpoint, change its underlying HTTP Basic Auth definition's password in Dashboard (username zato.pubsub.demo.secdef)

  • Afterwards, the endpoint can be used for demo purposes using the REST calls described in this chapter