Zato 3.1 includes new means to manage access to REST services based on input Method and Accept headers in HTTP requests - here is how they can be employed in practice.

A bit of background

Prior to Zato 3.1, one could always build a REST API reacting to individual HTTP verbs by implementing handle_VERB methods in services, such as:

class MyService(Service):

    def handle_GET(self):
        # Reacts to GET requests

    def handle_POST(self):
        # Reacts to POST requests

    # Any other handle_VERB method will be used accordingly

This works and will continue to work as expected in all future Zato versions.

However, one aspect of it is that, if one uses SimpleIO, keeping all handler methods in the same service means that all of them share the same SIO definition which is not always desirable - for instance, input to POST may be unrelated to input that DELETE receives.

REST channel URL paths

In Zato 3.1 and newer, it is possible to create REST channels that have the same URL path but different services mounted on each channel, separately for each HTTP verb requires.

That is, whereas previously it was a single service with multiple handle_VERB methods, now it can be a set of services, each reacting to a different HTTP verb and all of them mounted on the same URL path.

In a way, this was supported previously but, if handle_VERB methods were not used, URL paths had to be distinct, e.g.

GET /api/user
DELETE /api/user/delete
POST /api/user/create

In 3.1+, this can be simplified to:

GET /api/user
DELETE /api/user
POST /api/user

Now, each of the combination of verb + path may be unique for a REST channel while previously each channel needed to have its own URL path.

Moreover, because each channel may have its own separate service, it also means that each service may have its own SimpleIO definition and that the service becomes less tied to REST.

HTTP Accept headers

This is a completely new feature in 3.1 which lets one have distinct REST channels depending on the requests's Accept headers.

For instance, let's say that we would like to process incoming invoices under POST /api/invoice but we would like to have two services reacting to the same endpoint, one for JSON and now for PDF invoices.

This can be achieved by configuring HTTP Accept headers in their channels, as below - note that the method and URL path are the same in both cases yet HTTP Accept and services are different because each service reacts to a different value of HTTP Accept:

HTTP Accept header patterns

We can go one better and take advantage of Accept header patterns - with an asterisk meaning any character - this will configure the channel to process requests matching any value that fits in with the pattern, e.g. text/* will mean text/csv, text/xml or anything that starts with text/.

However, seeing as it can be a number of input MIME types, at this point we may need to know what the actual value was - this can be extracted from the WSGI environment via self.wsgi_environ.


Zato 3.1 saw improvements and backward-compatible changes to how REST channels can be built.

It will now support more use-cases, such as single URL path channels with different HTTP verbs and independent SimpleIO definitions or HTTP Accept headers different for each channel.

In turn, this lets one build REST APIs that are more flexible and elastic in their design to react to different input criteria.

A feature new in Zato 3.1 is the ability to connect to LDAP servers, including Active Directory instances, and this article covers basic administration as programming tasks involved in their usage from Python code.

Creating connections

Connections can be easily created in web-admin. Navigate to Connections -> Outgoing -> LDAP and then click Create a new connection.

The same form works for both regular LDAP and AD - in the latter case, make sure that Auth type is set to NTLM.

The most important information is:

  • User credentials
  • Authentication type
  • Server or servers to connect to

Note that if authentication type is not NTLM, user credentials can be provided using the LDAP syntax, e.g. uid=MyUser,ou=users,o=MyOrganization,dc=example,dc=com.

Right after creating a connection be sure to set its password too - the password asigned by default is a randomly generated one.


It is always prudent to ping a newly created connection to ensure that all the information entered was correct.

Note that if you have more than one server in a pool then the first available one of them will be pinged - it is the whole pool that is pinged, not a particular part of it.

Active Directory as a REST service

As the first usage example, let's create a service that will translate JSON queries into LDAP lookups - given username or email the service will basic information about the person's account, such as first and last name.

Note that the conn object returned by client.get() below is capable of running any commands that its underlying Python library offers - in this case we are only using searches but any other operation can also be used, e.g. add or modify as well.

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

# stdlib
from json import loads

# Bunch
from bunch import bunchify

# Zato
from zato.server.service import Service

# Where in the directory we expect to find the user
search_base = 'cn=users, dc=example, dc=com'

# On input, we are looking users up by either username or email
search_filter = '(&(|(uid={user_info})(mail={user_info})))'

# On output, we are interested in username, first name, last name and the person's email
query_attributes = ['uid', 'givenName', 'sn', 'mail']

class ADService(Service):
    """ Looks up users in AD by their username or email.
    class SimpleIO:
        input_required = 'user_info'
        output_optional = 'message', 'username', 'first_name', 'last_name', 'email'
        response_elem = None
        skip_empty_keys = True

    def handle(self):

        # Connection name to use
        conn_name = 'My AD Connection'

        # Get a handle to the connection pool
        with self.out.ldap[conn_name].conn.client() as client:

            # Get a handle to a particular connection
            with client.get() as conn:

                # Build a filter to find a user by
                user_info = self.request.input['user_info']
                user_filter = search_filter.format(user_info=user_info)

                # Returns True if query succeeds and has any information on output
                if, user_filter, attributes=query_attributes):

                    # This is where the actual response can be found
                    response = conn.entries

                    # In this case, we expect at most one user matching input criteria
                    entry = response[0]

                    # Convert to JSON so it is easier to handle
                    entry = entry.entry_to_json()

                    # Load from JSON to a Python dict
                    entry = loads(entry)

                    # Convert to a Bunch instance to get dot access to dictionary keys
                    entry = bunchify(entry['attributes'])

                    # Now, actually produce a JSON response. For simplicity's sake,
                    # assume that users have only one of email or other attributes.
                    self.response.payload.message = 'User found'
                    self.response.payload.username = entry.uid[0]
                    self.response.payload.first_name = entry.givenName[0]
                    self.response.payload.last_name =[0]
           = entry.mail[0]

                    # No business response = no such user found
                    self.response.payload.message = 'No such user'

After creating a REST channel, we can invoke the service from command line:

$ curl "localhost:11223/api/get-user?user_info=MyOrganization\\MyUser" ; echo
    "message": "User found",
    "username": "MyOrganization\\MyUser",
    "first_name": "First",
    "last_name": "Last",    
    "email": ""

Checking user credentials

A recurrent task in enterprise integrations in checking user credentials on behalf of systems that are not able to connect to AD or LDAP themselves; for instance, because they do not support the LDAP protocol or because a particular architecture disallows for them to make direct connections to backend servers.

To support this use-case, a separate method was added to the Python API specifically to validate user credentials - the code below is everything that is needed to confirm if user credentials are valid:


def handle(self):

    # Connection name to use
    conn_name = 'My AD Connection'

    # Credentials to check
    username = 'myuser'
    password = 'mypassword'

    # Get a handle to the connection pool object
    with self.out.ldap[conn_name].conn.client() as client:

        # Check credentials using the pool's configuration
        is_valid = client.check_credentials(username, password)

        if is_valid:
            # Credentials are valid, act accordingly here

            # Invalid username or password, return an error here


Full support for LDAP and Active Directory connections was added in Zato 3.1 and the Python API exposed grants one access to all the operations possible - offering means to integrate with directories or making them communicate with other technologies or protocols is now just a matter of authoring a service and exposing it through a channel, such as REST or one of the other types that Zato supports.

Zato has had support for FTP/FTPS since its inception. In Zato 3.1+, SFTP is also an option to consider for file transfer and this post offers an introduction to the functionality.


For build and deployment automation, zato enmasse is the command line tool most convenient to use, but during initial development SFTP connections can be constructed in web-admin.

The form lets one provide all the default options that apply to each SFTP connection - remote host, what protocol to use, whether file metadata should be preserved during transfer, logging level, bandwidth limit for each connection, SSH identity and config files as well as additional SSH options - the last one means that any SSH option that man sftp lists can also be used in Zato connections.


The first thing that one can do right after the creation of a new connection is to ping it, to check if the server is responding.

Pinging opens a new SFTP connection and runs the ping command - in the screenshot above it was ls . - a practically no-op command whose sole purpose is to let the connection confirm that commands in fact can be executed, which proves the correctness of the configuration.

This will either returns details of why a connection could not be established or the response time if it was successful.

Rapid development

Having validated the configuration by pinging it, we can now execute SFTP commands straight in web-admin from a command shell:

Any SFTP command, or even a series of commands, can be sent and responses retrieved immediately. It is also possible to increase the logging level for additional SFTP protocol-level details.

This makes it possible to rapidly prototype file transfer functionality as a series of scripts that can be next moved as they are to Python-based services.

Python API

For Python services, an extensive API is available. The API can execute transfer commands individually or in batches but alternatively it may make use of SFTP scripts previously created in web-admin. Here is how it can be used in practice:

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

# Zato
from zato.server.service import Service

class MySFTPService(Service):
    def handle(self):

        # Connection to use
        conn_name = 'My SFTP Connection'

        # Get a handle to the connection object
        conn = self.out.sftp[conn_name].conn

        # Execute an arbitrary script with one or more SFTP commands, like in web-admin
        my_script = 'ls -la /remote/path'

        # Ping a remote server to check if it responds

        # Download an entry, possibly recursively'/remote/path', '/local/path')

        # Like .download but remote path must point to a file (exception otherwise)
        conn.download_file('/remote/path', '/local/path')

        # Makes the contents of a remote file available on output
        out ='/remote/path')

        # Uploads a local file or directory to remote path
        conn.upload('/local/path', '/remote/path')

        # Writes input data out to a remote file
        data = 'My data'
        conn.write(data, '/remote/path')

        # Create a new directory

        # Create a new symlink

        # Create a new hard-link

        # Delete an entry, possibly recursively, no matter what kind it is

        # Like .delete but path must be a directory

        # Like .delete but path must be a file

        # Like .delete but path must be a symlink

        # Get information about an entry, e.g. modification time, owner, size and more
        info = conn.get_info('/remote/path')

        # A boolean flag indicating if path is a directory
        result = conn.is_directory('/remote/path')

        # A boolean flag indicating if path is a file
        result = conn.is_file('/remote/path')

        # A boolean flag indicating if path is a symlink
        result = conn.is_symlink('/remote/path')

        # List contents of a directory - items are in the same format that .get_info uses
        items = conn.list('/remote/path')

        # Move (rename) remote files or directories
        conn.move('/from/path', '/to/path')

        # An alias to .move
        conn.rename('/from/path', '/to/path')

        # Change mode of entry at path
        conn.chmod('600', '/path/to/entry')

        # Change owner of entry at path
        conn.chown('myuser', '/path/to/entry')

        # Change group of entry at path
        conn.chgrp('mygroup', '/path/to/entry')


SFTP are a new file transfer option added in Zato 3.1. Users may quickly prototype SFTP scripts in web-admin and employ them in Zato services. Alternatively, a full Python API is available for programmatic access to remote file servers. Combined, the features make it possible to create scalable and reusable file transfer services in a quick and efficient manner.

This post provides steps needed to enable and make use of TOTP two-factor authentication in Zato web-admin - the method is compatible with the most popular apps such as Google Authenticator, Authy or FreeOTP.

The functionality is slated for release in Zato 3.1 and, currently, it is available via Zato source installation.

Modifying web-admin.conf

We start by letting web-admin know that it is to require TOTP security codes from users. This is a global flag for all users.

  • Stop web-admin
  • Open web-admin.conf file
  • Set "is_toptp_enabled" to true (note that it is lower-case, without quotes)
  • Save the file
  • Start web-admin back

Generating the initial TOTP key

Now that the two-factor authentication with TOTP is enabled, we need the initial TOTP key for each user, otherwise they will not be able to log in at all.

Command line can be used to reset such keys for any user - in this case the process of resetting a key will amount to setting the initial one, as in the sample below that resets the key for a web-admin user called admin:

$ zato reset-totp-key /path/to/web-admin admin

Such a key can now be transferred to an app of choice to generate time-based security codes that web-admin will require.

Logging in

The app that the key was saved to will now present security codes needed for logging in.

Each such key is valid for at most 30 seconds - in this way a potential attacker will have to take two factors into account, one is your password and the other is getting access to your app and the associated TOTP secret key. Replay attacks are not particularly feasible against such codes because they will change frequently.

As always, even without TOTP, you should also make sure you rate-limit login attempts to web-admin by configuring your frontend load-balancer, or proxy, accordingly. With TOTP, since the codes are relatively short, it would be possible for an attacker to check them all very quickly so this kind of rate-limiting is of double importance.

Changing your TOTP key in web-admin

Having logged in, it is possible to generate secret keys directly in web-admin too. This option will also output QR codes to transfer the code to mobile devices simply by scanning the resulting QR code.


That is all - everything is set up and two-factor authentication will be enforced each time anyone logs in. TOTP is easy to configure and use so it is recommended to enable it as an additional security layer.

As of today, everyone building Zato from source will notice a significant decrease in time needed for servers to start and stop - it is now possible for an entire server to boot up to a fully functional state in less than two seconds and to shut down in half a second.

The changes in startup time were possible thanks to modifications to how services are deployed - deploying 700 services a second using just one CPU is now feasible, as in the snippet below.

Note that it means services that are actually ready to accept and process incoming messages, in this particular case there were almost 600 internal services and 100+ user-defined ones.

It is easiest to observe it when a server is started in foreground, using the --fg switch; after the initial messages comes the part when the server deploys all of its services, in this case it took roughly 0.9 s.

$ zato start ./server1 --fg
2019-02-18 11:36:07,394 - Starting gunicorn 19.9.0
2019-02-18 11:36:07,396 - Listening at: (4640)
2019-02-18 11:36:07,396 - Using worker: gevent
2019-02-18 11:36:07,398 - Booting worker with pid: 4662

2019-02-18 11:36:08,462 - Deploying cached internal services (server1)
2019-02-18 11:36:09,229 - Deployed 558 cached internal services (server1)
2019-02-18 11:36:09,230 - Deploying user-defined services (server1)
2019-02-18 11:36:09,365 - Deployed 122 user-defined services  (server1)

Similarly, to stop a server, a fraction of a second is needed. Here, below, the server was running in foreground so Ctrl-C sufficed to stop it - and it took 0.3 s for it to shut down.

2019-02-18 11:46:22,060 - gunicorn.main:271 - Handling signal: int
2019-02-18 11:46:22,060 - Closing IPC (/pubsub/pid)
2019-02-18 11:46:22,061 - Closing IPC (/connector/config)
2019-02-18 11:46:22,075 - Stopping server process (server1:4662) (4662)
2019-02-18 11:46:22,163 - Process exiting (pid: 4662)
2019-02-18 11:46:22,361 - Shutting down: Main

All of it will be released in June 2019, as part of Zato 3.1, and today - one can simply install Zato from source and take advantage of the improvements already.