Please read the high-level overview if you have not done it already for an introduction of how Zato and Python help with complex and advanced systems integrations.
This tutorial will guide you through the process of creating a real-world API service that, with Python, will integrate three applications using REST and AMQP. The result will be a solution ready to use in production.
But first, here are some real-world examples of what Zato can be used for:
What they all have in common is the level of complexity involved in integration processes. It is typical for such environments to have at least several and sometimes several dozen or several hundred independent applications exchanging messages, each possibly using a different technology and data format.
Note that Zato is a general-purpose platform and the samples span various distinct industries and domains. The platform can be used in all kinds of scenarios that require automation and integrations of distributed systems.
If Zato is a platform for API services then what is a service?
In essence, a service is a piece of server-side functionality carrying out a business action useful and interesting to applications that would like to use it. This is a high-level definition and this is the correct way to think about it.
It all stems from the fact that you should think about the systems and applications comprising your architecture in terms of consumers and providers of services. Each part of the architecture offers services to other parts and, conversely, each part consumes services offered by other elements of the architecture. Taken together, services form higher-level processes fulfilling business needs set before the IT architecture.
Zato is a multi-protocol platform and services are often not tied to any specific protocol. It means that it is possible to design services that can be invoked through REST but they can also listen for data from AMQP, IBM MQ queues or SQL databases. They can also accept HL7 MLLP, SOAP, WebSocket, SFTP, FTP, e-mail, JSON-RPC and ZeroMQ-based messages.
Naturally, REST is ubiquitous and usually, this is the way that most APIs are exposed through but there are other ways too and in various scenarios other means of communication are employed.
Zato ships with connectors and adapters for REST, AWS S3, AMQP, Kafka, MongoDB, Redis, HL7, Odoo, SAP, IBM MQ, SQL, SOAP, FTP, SFTP, LDAP, Cassandra, Dropbox, Twilio, IMAP, SMTP, ElasticSearch, Solr, Swift, Slack, Telegram, WebSockets and ZeroMQ. Because it is written in Python, you have access to many third-party libraries which provide connectivity to other types of systems.
Because API platforms often need dashboards, it is also possible to use Django templates with Zato to output user interfaces.
Built-in security options include API keys, Basic Auth, JWT, NTLM, OAuth, RBAC, SSL/TLS, Vault, WS-Security and XPath. It is always possible to secure services using other, non-built in, means.
In terms of its implementation, an individual Zato service is a Python class implementing a specific method called self.handle. The service receives input, processes it according to its business requirements, which may involve communicating with other systems, applications or services, and then some output is produced. Note that both input and output are optional, e.g. a background service transferring files between applications will usually have neither whereas a typical CRUD service will have both.
Because a service is merely a Python class, it means that each one consumes very little resources and it is possible to deploy hundreds or thousands of services on a single Zato server. And because Zato can use multiple CPUs and multiple Linux instances, it scales without limits both horizontally and vertically.
Services accept their input through channels - a channel tells Zato that it should make a particular service available to the outside world using such and such protocol, data format and security definition. For instance, a service can be mounted on independent REST channels, sometimes using API keys and sometimes using Basic Auth. Additionally, each channel type has its own specific pieces of configuration, such as caching, timeouts or other options.
Services can invoke other Zato services too - this is just a regular Python method call, within the same Python process. It means that it is very efficient to invoke them - it is simply like invoking another Python method.
Services are hot-deployed to Zato servers without server restarts and a service may be made available to its consumers immediately after deployment.
There are plugins for Visual Studio Code and PyCharm that automatically deploy your service each time you save it in your IDE. Other code editors or IDEs can be used too.
During development, usually, the built-in web-admin dashboard is used to create and manage channels or other Zato objects. As soon as a solution is ready for DevOps automation, a configuration of a solution can be deployed automatically from the command line or directly from a git clone which makes it easy to use Zato with tools such as Terraform, Nomad or Ansible.
After completing the tutorial, we will have:
We will be implementing an API integration process typical to banks and other financial institutions.
The Client App is a building block that we will not be developing in the tutorial - this is where Django, React, Vue, Flutter, ASP.NET and other frameworks can be used in actual projects.
On the other hand, remember that other backend systems can invoke the service too - this is crucial, the same one can be made available to many applications, each with its own access channel, even if in the tutorial we will assume there is only one API client.
The installation procedure for Mac, Linux, Docker and Vagrant is covered in the other version of the tutorial.
Under Windows, follow the steps below. Administrator rights are not required.
Installing Zato creates a quickstart cluster as well. Quickstart clusters are a convenient way of creating new, self-contained, fully functional environments comprised of one server, dashboard and a scheduler.
You can create quickstart clusters from the command line and via Docker Quickstart as well and they can be used for any purposes, from development, through testing to production.
The two main endpoints of a Windows quickstart cluster are:
Address | Purpose | Notes |
---|---|---|
localhost:8183 | Dashboard | A Web-based Dashboard to manage Zato servers. The username is "admin" and how to reveal the auto-generated password is described below. |
localhost:17010 | Server | A Zato API server instance. This is the main component that runs your API integrations and backend Python services. |
As a rule, there are no default credentials anywhere in Zato, which means that each environment has different, auto-generated passwords.
To retrieve the one generated for you, click Programs → Zato 3.2 → Show configuration
in the Windows Start menu.
This will open a JSON file with various details of the environment, including two keys that are of interest now:
Key | Notes |
---|---|
dashboard_password | Used to log in to the Dashboard at localhost:8183. Username: "admin". |
ide_password | Used when configuring a VS Code or PyCharm plugin for Zato. Username: "ide_publisher". |
Note that you can also change the passwords at any time directly in the JSON file and they will be used when you restart the environment by closing its window and clicking "Start Zato" again.
You can now go to the Dashboard at http://localhost:8183 and log in with user 'admin' and password from the configuration file.
Zato itself and the newly created environment is installed to your Windows user's ~\AppData\Local
directory.
The exact name of this directory may differ depending on the locale set for your Windows system. If the language is English
and the username is "dsuch" then the default,
full path will be C:\Users\dsuch\AppData\Local\Zato3.2
, as in the screenshot below.
Note that, because Zato does not require Administrator permissions, it never installs anything to "C:\Program Files" or similar locations.
You can author Zato services with any code editor and you can also install a plugin for your IDE to auto-deploy your services each time a Python file is saved on disk.
IDE | Notes |
---|---|
![]() |
Configure Visual Studio Code for work with Zato. |
![]() |
Configure PyCharm for work with Zato. |
Depending on the plugin's version, it may by default try to connect to localhost:11223 instead of localhost:17010
because 11223 is the default plugin port under Mac, Linux and Docker.
If that be the case, make sure to configure the plugin to use localhost:17010
which is what Zato for Windows uses.
Regardless of what your IDE and plugin are, the username for the plugin to authenticate with a Zato server is always "ide_publisher".
Hot-deployment is a key concept in Zato. The term means the process of transferring your service to a cluster.
It is considered hot
because, afterwards, it does not require server restarts, i.e. you hot-deploy a service
and it is immediately available on all the servers.
If there is more than one server in the cluster, it suffices to hot-deploy the service to only one of them and it will synchronize with other nodes in the cluster.
There are a few ways to hot-deploy services. The first two will be used in the tutorial but we will describe each to let you understand what the options are and when to use them.
From your IDE
Commonly used during development - once you install a plugin for the IDE, each time you press Ctrl-S to save a service on disk, it will be auto-deployed to your cluster and made available for immediate use.
Command line
This is used for deployment automation or if you have an IDE or editor without a Zato plugin.
Each Zato server monitors a specific directory, called a hot-deploy directory, and each time Python files with your services are saved there, that server will pick them up and hot-deploy throughout the cluster.
In the quickstart cluster from this tutorial, the directory is ~\AppData\Local\Zato3.2\env\qs-1\server1\pickup\incoming\services
.
During development, you can save your files with Zato services directly in this directory and then, when you press Ctrl-S, the file will be deployed to the cluster. You can also clone your git repository directly into this directory.
Another way to use it during development is to make it point to a git clone residing in another directory and again, each time you save a file its contents are sent to all the servers.
This method is used for automation too - simply use built-in copy
or robocopy
Windows tools to copy files into
the directory and all the services from these files will be deployed.
Dashboard
When you log in to the Dashboard and navigate to Services, you will note a button called "Upload services". This will let you deploy local files to a remote server. This is useful when there is no direct connection to the server, e.g. no way to ssh into it.
Local config file
This method is usually used for automated deployments only - it lets you point a starting server to files from the file system that it should deploy.
The difference between it and a hot-deploy directory is that the latter requires for the server to be already running whereas this one tells a server what it should do while it is still starting up.
This option is most often employed when building one's own Docker images or using Terraform, Packer and similar tools.
Remote file transfer
This automation method uses file transfer to let servers listen for changes in directories of remote servers.
For instance, you can have a central git clone of a repository for multiple environments and Zato servers will connect to it via SFTP, download any new or changed ones and deploy them locally.
In terms of the end result, there is no difference between the methods, they achieve exactly the same result.
This is actually a good example of the way Zato itself is designed around reusable services - all these deployment methods, all these channels, ultimately lead to the same services that deploy your code and it is only the manner in which they are accessed that differs.
We can now create the first service and hot-deploy it. Create a new file called api.py with the contents below. This is a basis of the service that we will fill in with details later on.
# -*- coding: utf-8 -*-
# zato: ide-deploy=True
from zato.server.service import Service
class GetUserDetails(Service):
""" Returns details of a user by the person's ID.
"""
name = 'api.user.get-details'
def handle(self):
# For now, return static data only
self.response.payload = {
'user_name': 'John Doe',
'user_type': 'SRT'
}
If you configured a plugin for PyCharm or Visual Studio Code, note the highlighted line - this is a special marker which lets the plugin know that saving this file should result in the IDE's deploying it to your cluster.
Without a plugin, you need to save the file in the server's ~/env/qs-1/server1/pickup/incoming/services directory.
If you want to deploy it to a Zato server from your browser, log in to the Dashboard, go to Services and click Upload package. Dashboard's address is localhost:8183, username is "admin" and the password is in your Start menu, under Programs → Zato 3.2 → Show configuration.
No matter how you deploy the service, there will be activity in the server's log:
Having deployed the code, we can confirm in the Dashboard that the service is there.
Once you logged in to the Dashboard, navigate to Services. Enter "get-details" in the search box, then click Show services. Click the service's name and this will display basic information about the deployed service. You can click Source code to confirm that this is the same service.
We have a service so now we can create a REST channel for it.
We want to invoke our API service using REST but we also want to make sure that access to it is secured so we will first create a security definition for our API client.
In Dashboard, go to Security → Basic Auth → Click "Create a new definition" and enter:
Clicking OK will create the definition with its user's password automatically set to a random uuid4 so we need to reset it by clicking "Change password" and providing a new one - it is up to you to decide what it should be.
Now, we can create a REST channel by going to Connections → Channels → REST, as below:
Click Create a new REST channel link:
Fill out the form as here, the fields to provide values for are:
Clicking OK will create the channel and we will be able to invoke the API service now.
We are going to use curl to invoke the service - we will access it through the server's port of 17010, as below. Note that you need to enter the API client's password too.
$ curl http://api:<password-here>@localhost:17010/api/v1/user ; echo
{"user_name":"John Doe","user_type":"SRT"}
$
Everything works as expected - you have just created and invoked your first API service! Now, try to see what happens if you provide an invalid password or a URL path - your requests will not be allowed.
This concludes the first part of the tutorial and the next one will see the service integrate with external systems to transform and enrich their replies before producing the final response to the API client.
But first, there is one observation to be made - the quickstart cluster that you created is a real, fully functional environment. If you were to create it from scratch, by adding each component individually, Dashboard, server and scheduler, the outcome would be precisely the same.
In other words, quickstart clusters are a convenient method for the creation of new environments and they can be very well used not only for development but for testing and production too.
Now, we are ready to go to the second part of the tutorial.