This article offers a high-level overview of the public services that Zato offers to users wishing to manage their environments in an API-driven manner in addition to web-admin and enmasse tools.

Overview

Most users start to interact with Zato via its web-based admin console. This works very well and is a great way to get started with the platform.

In terms of automation, the next natural step is to employ enmasse which lets one move data across environments using YAML import/export files.

The third way is to use the API services - anything that can be done in web-admin or enmasse is also available via dedicated API services. Indeed, both web-admin and enmasse are clients of the same services that users can put to work in their own integration needs.

The public API is built around a REST endpoint that accepts and produces JSON. Moreover, a purpose-built Python client can access all the services whereas an OpenAPI-based specification lets one generate clients in any language or framework that supports this popular format.

Python usage examples follow in the blog post but the full documentation has more information about REST and OpenAPI too.

Prerequisites

First thing needed is to set a password for the API client that will be used, it is an HTTP Basic Auth definition whose username is pubapi. Remember, however, that there are no default secrets in Zato ever so the automatically generated password cannot be used. To change the password, navigate in web-admin to Security -> HTTP Basic Auth and click Change password for the pubapi user.

Now, we can install the Python client package from PyPI. It does not matter how it is installed, it can be done under a virtual environment or not, but for simplicity, let's install it system-wide:

$ sudo pip install zato-client

This is it as far as prerequisites go, everything is ready to invoke the public services now.

Invoking API services

For illustration purposes, let's say we would like to be able to list and create ElasticSearch connections.

The easiest way to learn how to achieve it is to let web-admin do it first - each time a page in web-admin is accessed or an action like creating a new connection is performed, one or more entries are stored in admin.log files on the server that handles the call. That is, admin.log is the file that lists all the public API services invoked along with their input/output.

For instance, when you list ElasticSearch connections, here is what is saved in admin.log:

INFO - name:`zato.search.es.get-list`, request:`{'cluster_id': 1}`
INFO - name:`zato.search.es.get-list`, response:`'
   {"zato_search_es_get_list_response": [],
   "_meta": {"next_page": null, "num_pages": 0, "prev_page": null,
   "has_prev_page": false,
   "cur_page": 1, "page_size": 50, "has_next_page": false, "total": 0}}'

It is easy to discern that:

  • The service invoked was zato.search.es.get-list
  • Its sole input was the cluster ID to return connections for
  • There were no connections returned on output which makes sense because we have not created any yet

Let's do the same in Python now:

# Where to find the client
from zato.client import APIClient

# Credentials
username = 'pubapi'
password = '<secret>'

# Address to invoke
address = 'http://localhost:11223'

# Build the client
client = APIClient(address, username, password)

# Choose the service to invoke and its request
service_name = 'zato.search.es.get-list'
request = {'cluster_id':1}

# Invoke the API service
response = client.invoke(service_name, request)

# And display the response
print(response.data)

Just like expected, the list of connections is empty:

$ python pubapi.py 
[]
$ 

Navigate to web-admin and create a new connection via Connections -> Search -> ElasticSearch, as below:

Let's re-run the Python example now to witness that the newly created connection can in fact be obtained from the service:

$ python pubapi.py 
[{
  u'name': u'My Connection',
  u'is_active': True,
  u'hosts': u'127.0.0.1:9200\r\n',
  u'opaque1': u'{}',
  u'timeout': 5,
  u'body_as': u'POST',
  u'id': 1
}]
$ 

But this is not over yet - we still need to create a new connection ourselves through an API service. If you kept admin.log opened while the connection was being created in web-admin, you noticed that the service to do it was called zato.search.es.create and that its input was saved to admin.log too so we can just modify our Python code already:

# Where to find the client
from zato.client import APIClient

# Credentials
username = 'pubapi'
password = '<secret>'

# Address to invoke
address = 'http://localhost:11223'

# Build the client
client = APIClient(address, username, password)

# First, create a new connection
service_name = 'zato.search.es.create'
request = {
    'cluster_id':1,
    'name':'API-created connection',
    'hosts': '127.0.0.1:9201',
    'timeout': 10,
    'body_as': 'POST'
}
client.invoke(service_name, request)

# Now, get the list of connections, it should include the newly created one
service_name = 'zato.search.es.get-list'
request = {'cluster_id':1}
response = client.invoke(service_name, request)

# And display the response
print(response.data)

This is a success again because on output we now have both the connection created in web-admin as well as the one created from the API client:

$ python pubapi.py 
[{
 u'name': u'API-created connection',
 u'is_active': True,
 u'hosts': u'127.0.0.1:9201',
 u'opaque1': u'{}',
 u'timeout': 10,
 u'body_as': u'POST',
 u'id': 2
},
{
 u'name': u'My Connection',
 u'is_active': True,
 u'hosts': u'127.0.0.1:9200\r\n',
 u'opaque1': u'{}',
 u'timeout': 5,
 u'body_as': u'POST',
 u'id': 1
}]
$ 

Just to double-check it, we can also list the connections in web-admin and confirm that both are returned:

Summary

That is really it. The process is as straightforward as it can get - create a client object, choose a service to invoke, give it a dict request and a Python object is returned on output.

Note that this post covered Python only but everything applies to REST and OpenAPI-based clients too - the possibilities to interact with the public API are virtually limitless and may include deployment automation, tools to test installation procedures or custom command and control centers and administration dashboards.

This post describes a couple of new techniques that Zato 3.0 employs to make API servers start up faster.

When a Zato server starts, it carries out a series of steps, one of which is deployment of internal API services. There are 550+ of internal services, which means 550+ of individual features that can be made use of - REST, publish/subscribe, SSO, AMQP, IBM MQ, Cassandra, caching, SAP Odoo, and hundreds more pieces are available.

Yet, what internal services have in common is that they change relatively infrequently. They do change from time to time but this does not happen very often. This realization led to the creation of a start-up cache of internal services.

Auto-caching on first deployment

Observe the output when a server is started right after installation, with all the internal services about to be deployed along with some of the user-defined ones.

In this particular case, the server needed around 8.5 second to deploy its internal services but while it was doing it, it also cached them all for later use.

Now, when the same server is stopped and started again, the output will be different. Nothing changed as far as user-defined services go but things changed with regards to the internal ones - not only did the server deploy the internal services but it also did it by re-using the cache created above and, consequently, 3 seconds were needed to deploy them.

Such a cache of internal services is created and maintained by Zato automatically, no user action is required.

Disabling internal services

Auto-caching is already a nice improvement but it is possible to go one better. By default, servers deploy all of the internal services that exist - this is because users may want to choose in their projects any and all of the features that the internal services represent.

However, in practice, most projects will use a select few technologies, e.g. REST and AMQP, or REST, IBM MQ, SAP and ElasticSearch, or any other combination, but not all of what is possible.

This explains the addition of a new feature which allows one to disable all the internal services that are known not to be needed in a particular project.

When you open a given server's server.conf file, you will find entries in the [deploy_internal] stanza whose subset is below. Note that if your Zato 3.0 version does not have it, you can copy the stanza over from a newly created server.

The list contains not internal services as such but Python modules to which the services belong, each module concerns a particular feature or technology, AMQP, JMS IBM MQ, WebSockets, Amazon S3 and anything else. Thus, if something is not needed, you can simply change True to False for each module that is not used.

But, you need to keep in mind that all the internal services were already cached before so, having changed True to False in as many places as needed, we also need a way to recreate the cache.

This is done by specifying the --sync-internal flag when servers are started; observe below what happens when some of the internal services were disabled and the flag was provided.

All the user-defined services deployed as previously but the cache for the internal ones was recreated and only some of them were deployed, only the ones that were needed in this particular project, which happens to primarily include REST, WebSockets, Vault and publish/subscribe.

Note that even without the cache, the server needed only 4.1 second to deploy internal services which neatly dovetails with the fact that previously it needed 8.5 to deploy roughly twice as many of them.

This also means that with the cache already in place, the services will be deployed even much faster, which is indeed the case below. This time the server deployed the internal services needed in this project in 1.3 second, which is much faster than the original 8.5 second.

This process can be applied as many times as needed, each time you need new functionality disabled or enabled, you just edit server.conf, restart servers and that is it, the caches will be populated automatically.

With some of the services disabled, a caveat is that parts of web-admin will not be able to list or manage connections whose backend services were taken out but this is to be expected, e.g. if FTP connections were disabled in server.conf then it will not be possible to access them in web-admin.

One final note is that --sync-internal should really only be used when needed. The rationale behind the start-up cache is to make the process faster so this flag should not be used all the time, rather, there are two cases where it needs to be used:

  • When changing which internal services to deploy, as detailed in this post
  • When applying updates to your Zato installation - some of the updates may change, delete or add new internal services, which is why the caches need to be recreated in such cases

Version 1.12 of zato-apitest 1.12 has just been released. This version simplifies installation requirements and adds compatibility with PostgreSQL 10+ databases.

zato-apitest is an API testing tool designed from ground up with convenience and ease of use in mind. It supports REST, SQL, Cassandra and Zato-based APIs with tests written in plain English.

There is no need for manual programming, though if required, it is easy to extend it in Python.

It ships with a built-in demo; right after the installation, run apitest demo and a sample test case will be set up and run against a test server, as below:

$ sudo pip install zato-apitest
$ apitest demo

Screenshots

The tool is part of the Zato API and backend server platform but can be used standalone with or without Zato services. More information, including documentation and usage examples can be found here.

Zato-based WebSockets are a great choice for high-performance API integrations. WebSockets have minimal overhead, which, coupled with their ability to invoke services in a synchronous manner, means that large numbers of clients can easily connect to Zato API servers.

Introduction

The crucial distinction between WebSockets and typical REST-based APIs is that clients based on the former protocol always establish long-running TCP connections and, once connected, the overhead they incur is practically negligible.

With a great number of clients a series of questions naturally appears. What are the clients currently connected? What if I want to force one to disconnect? What topics and message queues are they subscribed to? How can I communicate with the WebSockets directly from web-admin?

This blog post answers all these questions and then some more.

WebSocket channels

As a refresher, recall that all WebSocket clients connect to Zato through their channels. Each channel encapsulates basic information about what is expected from each client, e.g. their credentials or which service is responsible for their requests.

Screenshots

Listing connections

With a desired channel in place, we can start a few clients and then go straight to the listing of connections, as in the screenshow below:

Screenshots

Screenshots

By default, all connections for a given channel are listed but it is possible to filter them out by external client ID - each WebSocket identifies with a unique client ID, as assigned by the system on whose behalf the WebSocket connects. This makes it easy to find connections even if they go through a series of networks.

Each WebSocket is identified by a series of attributes, Client, Remote, Local and Connection time.

Each Client connection has a few of identifiers:

  • Unique connection ID assigned by Zato, changed each time a client connects
  • Client ID - unique ID assigned by the remote end, persists across connections
  • Client name - similar to Client ID but there is no requirement that it be unique

Remote TCP end has two attributes:

  • IP address as observed from a Zato server's perspective
  • FQDN (domain name) of that IP address

Local server to which a WebSocket is connected:

  • Its IP address and port number
  • Server name and server process ID (PID) to which the WebSocket is attached

Connection time is by default presented in current user's timezone but clicking it changes the format to UTC.

Checking pub/sub subscriptions

Screenshots

WebSockets may participate in pub/sub processes and it is possible to look up all the topics a particular connection is subscribed to. Note that subscription times may predate connection times - this will be the case if a WebSocket connects, subscribes to a topic, then disconnects and connects again. In such a case, the subscription time will be earlier than the last connection time.

Invoking WebSockets directly

Screenshots

It is possible to send requests straight to a WebSocket, waiting up to timeout seconds for the reply. This lets one communicate with the remote connection directly, which is of great assistance in many low-level diagnostic scenarios.

Disconnecting API clients

Screenshots

Each WebSocket can be disconnected separately - on the protocol level, it will send a Close event to the remote end, afterwards cleaning up all the internal resources taken up by the connection.

Summary

API integrations with WebSockets offer an alternative to REST whose greatest advantage is reduced runtime processing overhead. Zato offers built-in GUI tools to create and manage WebSockets, including searching, listing and direct communication with each WebSocket straight from a browser's window.

To learn more about how to integrate APIs with Zato, visit the tutorial and downloads sections of the extensive documentation which cover everything needed to get started with the platform.

Since version 3.0, it is possible to directly connect Zato clusters and exchange messages as though remote services where running in a local instance. This makes it an ideal choice for environments split into multiple parts.

Introduction

The reasons to have more than one cluster, each with one or more servers, may vary:

  • For HA and performance, environments may be broken out geographically into a setup with one cluster per continent or a region of the world
  • CPU-extensive operations may be carried out in one cluster with another making use of the results the former produces to offer a set of APIs
  • For legal reasons, it may not be allowed to run all integration services in one cluster, using the same hardware and software infrastructure

The new feature in Zato 3.0 which allows for efficient communication between clusters are WebSocket connections - one of clusters will create a channel through with other clusters may invoke its services via their outgoing connections.

WebSockets (WSX for short) have essentially no overhead in practice but they can be used for bi-directional communication hence they are a great choice for such scenarios.

From a Zato programmer's perspective, all the communication details are hidden and a couple of lines of code suffices to invoke services or receive messages from remote clusters, for instance:

# Obtain a handle to a remote connection
with self.out.wsx.get('My Connection').conn.client() as client:

    # Invoke a remote service - expects a Python dict on input
    # and returns a Python dict on response. All the serialization
    # and network connectivity is handled automatically.
    response = client.invoke(msg)

Architecture and configuration

Screenshots

  • Each cluster which is to become a recipient of messages from other clusters needs to have a new WebSocket channel created with service helpers.web-sockets-gateway mounted on it. A security definition should also be attached as required.

  • Each cluster that should invoke another one needs to have an outgoing WebSocket connection created - make sure Is remote end Zato checkbox is on and that credentials are provided, if required by the other side.

  • If the cluster with an outgoing connection is interested in receiving publish/subscribe messages, all topics it wants to subscribe to should be listed, one in each line. Make sure the cluster with a channel has a correct pub/sub endpoint configured for that channel.

  • The cluster which establishes the connection (here, cluster1) may also want to subscribe to events of interest via hooks services - more about it below.

  • Once an outgoing connection is created, internal tasks will start on cluster1 to establish a remote connection to server2. If successful, authentication will take place automatically. Finally, if configured, a hook service will fire to let cluster1 know that a new connection was established. Afterwards, cluster1 may start to invoke remote services.

  • There are no other steps involved, at this point everything is configured and ready to be used.

Screenshots

Screenshots

From a programmer's perspective

  • To invoke remote Zato services, programmers use WebSockets outgoing connections methods - providing a dictionary of input data to the invocation and receiving a dictionary of data on input. Note that the invocation is synchronous, your service is blocked until the remote cluster responds.
# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

from zato.server.service import Service

class MyService(Service):
    def handle(self):

        # Message to send - needs to be a dictionary with name
        # of the service to invoke as well as its input data, if any is required.
        # In this case, we are invoking an echo service
        # which writes back to output anything it receives on input.
        msg = {
             'service':'zato.helpers.echo',
             'request': {
                 'elem1': 'value1',
                 'elem2': 'value2',
             }
         }

        # Name of the connection to send messages through
        conn_name = 'My WSX Outconn'

        # Obtain a client from the connection pool
        with self.out.wsx.get(conn_name).conn.client() as client:

            # Send the message and read its response
            response = client.send(msg)

            # Or, client.invoke can be used with Zato WebSocket connections,
            # this method is an alias to client.send
            response = client.invoke(msg)

            # Log the response received
            self.logger.info('Response is `%s`', response.data)
INFO - Response is `{u'elem2': u'value2', u'elem1': u'value1'}`
  • To receive messages, hook services are used. There are three events for which hooks can be triggered - they can be handled by different services or the same one, it is up to users:
  1. Upon connecting to a remote cluster, including reconnects (on_connect)
  2. Upon receiving messages from remote clusters (on_message)
  3. Once a connection to the remote cluster is shut down (on_close)
  • The on_message hook can be combined with publish/subscribe topics and queues - each time the remote cluster (the one with a WSX channel) publishes a message that the local cluster (the one with a WSX outgoing connection) is interested in, the on_message hook will be called to handle it, in this manner making it possible for remote clusters to deliver messages to clusters subscribing to topics.

  • Each hook is just a Zato service with a specific SimpleIO signature, as in the on_message example below:

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

from zato.server.service import Opaque, Service

class OnMessageHook(Service):
    class SimpleIO:
        input_optional = (Opaque('ctx'),)

    def handle(self):

        # Object describing incoming data
        ctx = self.request.input.ctx

        # Message type
        msg_type = ctx.type

        # Data received
        data = ctx.data

        # Log message type
        self.logger.info('Msg type: `%s`', msg_type)

        # Log actual data
        self.logger.info('Data received: `%s`', data.data)

        # Log metadata - ID and timestamp
        self.logger.info('Meta: `%s` `%s`', data.id, data.timestamp)

Now, we can use web-admin to publish a test message and confirm that the on_message service receives it:

Screenshots

In the on_message service's server logs:

INFO - Msg type: `message`
INFO - Data received: `[
  {u'delivery_count': 0,
   u'msg_id': u'zpsme26726911ffbe8cba2cca278',
   u'expiration_time_iso': u'2086-09-21T14:03:05.285470',
   u'topic_name': u'/customer/new',
   u'pub_time_iso': u'2018-09-03T10:48:58.285470',
   u'priority': 5,
   u'expiration': 2147483647000,
   u'has_gd': True,
   u'data': u'This is a sample message',
   u'sub_key': u'zpsk.websockets.6ef529f7cab64a71d8bd2878',
   u'mime_type': u'text/plain',
   u'size': 24}
  ]`
INFO - `6fd296ecf78493a3a0ce7570` `2018-09-03T10:49:00.540024`

Summary