Employing JSON Schema is an easy and straightforward way to validate input to your APIs - learn below how it can be enabled in Zato 3.1+ services in one line of code.

Preparing a schema

Supposing that our service needs two parameters:

  • name (string)
  • pub_year (integer)

This is how a schema representing them may look like - let's save it in a file called myschema.json.

{
  "type": "object",
  "required": ["name", "pub_year"],
  "properties": {
    "name":  {"type" : "string"},
    "pub_year": {"type" : "number"}
  }
}

This is as simple as it can get - we list our parameters and for each one, we also say of what datatype they are.

Note that a JSON Schema may be of arbitrary complexity to express any kind of business needs. It is also possible to split bigger schemas into reusable pieces, ready for inclusion in more than one place.

Uploading the schema

Each server has a directory called config/repo/schema/json - this is where the schema needs to be uploaded.

For instance, if path to a server is /home/zato/env/server1 then the full path will be /home/zato/env/server1/config/repo/schema/json.

Save myschema.json to this directory and restart your server before continuing to the next step.

Configuring a service

To make use of the prepared schema, a service uses an attribute appropriately called schema, as below.

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

# Zato
from zato.server.service import Service

class MyService(Service):

    # Require JSON Schema validation
    schema = 'myschema.json'

    def handle(self):

        # If we are here, it means that the input schema validation succeeded
        self.logger.info("My request: %s", self.request.payload)

Note that the value of this attribute can be either relative or absolute. If it is relative, it is in relation to the main directory (config/repo/schema/json). If it is an absolute one, it must be a full path to the schema file in the filesystem.

Schema validation in runtime

You do not need to do anything else for schema validation to work, let's observe it.

As expected, on invalid input we get an error:

$ curl localhost:17010/schema/check -d '{"user_id":123}' ; echo
{"zato_env":
  {"result": "ZATO_ERROR",
   "cid": "0ee199609d5020d7cb8ca924",
   "details":
     {"is_ok": false,
      "cid": "0ee199609d5020d7cb8ca924",
      "message": "Invalid request"
  }}}
$

Whereas with correct data on input a message is stored in a log file:

INFO - My request: {'name':'The Garden of Cyrus', 'pub_year':1658}

Web-admin options

Sometimes it is handy to be able to disable input validation on demand, without redeployment of code. On the other hand, it is at times convenient to return more details than "Invalid request" alone.

This is exactly what can be done in web-admin, for each service separately:

Summary

That is all - you have just created a schema and enabled it for your service. Zato picked it up, started to enforce its definitions in runtime, and the whole of it took just a single line of code. Moreover, you can dynamically change the way it is used without making any updates to the source code.

This is a quick guide on how to turn SSH commands into a REST API service. The use-case may be remote administration of devices or equipment that does not offer a REST interface or making sure that access to SSH commands is restricted to selected external REST-based API clients only.

Python

The first thing needed is code of the service that will connect to SSH servers. Below is a service doing just that - it receives name of the command to execute and host to run in on, translating stdout and stderr of SSH commands into response documents which Zato in turn serializes to JSON.

It uses the sh library for SSH connections which means that each new one is started in a subprocess.

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

# stdlib
from traceback import format_exc

# sh
from sh import ssh

# Zato
from zato.server.service import Service

class SSHInvoker(Service):
    """ Accepts an SSH command to run on a remote host and returns its output to caller.
    """
    class SimpleIO:
        input_required = 'host', 'command'
        output_required = 'is_ok', 'cid'
        output_optional = 'stdout', 'stderr'
        skip_empty_keys = True
        response_elem = None

    def handle(self):

        # Local aliases
        host = self.request.input.host
        command = self.request.input.command

        # Correlation ID is always to be returned
        self.response.payload.cid = self.cid

        try:
            # Run the command and collect output
            output = ssh(host, command)

            # Assign both stdout and stderr to response
            self.response.payload.stdout = output.stdout
            self.response.payload.stderr = output.stderr

        except Exception:
            # Catch any exception and log it
            self.logger.warn('Exception caught (%s), e:`%s', self.cid, format_exc())
            self.response.payload.is_ok = False

        else:
            # Everything went fine
            self.response.payload.is_ok = True

Web-admin

In web-admin, let's go ahead and create an HTTP Basic Auth definition that a remote API client will authenticate against:

Now, the SSH service can be mounted on a newly created REST channel - note the security definition used and that data format is set to JSON.

Usage

At this point, everything is ready to use. For testing purposes, let's invoke the service from command line:

$ curl "api:8MbrHs2GHHQjw@localhost:11223/api/ssh" -d \
    '{"host":"localhost", "command":"uptime"}'
{
    "is_ok": true,
    "cid": "27406f29c66c2ab6296bc0c0",
    "stdout": " 09:45:42 up 37 min,  1 user,  load average: 0.14, 0.27, 0.18\n"}
$

This completes it - the service is deployed and made accessible via a REST channel that can be invoked using JSON. Any command can be sent to any host and, assuming that SSH commands can be executed at all, their output will be returned to API callers in JSON responses.

One of the newest additions in Zato 3.1 are MongDB connections - learn in this article how to create and use them in your Python API services.

Web-admin

As with most parts of Zato, the most straightforward way to create a new connection definition is to fill out a form in web-admin.

Note that MongoDB connections pack in a lot of options and, to keep the user interface as lightweight as possible, only the most commonly used ones are displayed in the default view and the rest needs to be toggled as needed.

Pinging

To confirm that the connection is configured correctly, including credentials to connect with, you can ping it.

If everything is set up correctly, a response such as the one below will be returned.

Python API

A sample Python service may look like below - the client returned by self.out.mongodb is an instance of pymongo.MongoClient so anything that the PyMongo library can do is achievable from Zato too.

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

# Zato
from zato.server.service import Service

class MyService(Service):
    def handle(self):

        # Obtain a MongoDB client
        client = self.out.mongodb['My MongoDB Connection'].conn.client

        # Select a database
        db = client.test

        # Insert a document
        db.my_collection.insert_one({'My object': 'My data'})

        # Get all documents already stored
        result = db.my_collection.find()

        # Print them out to logs
        for item in result:
            self.logger.info('Item: %s', item)

Now, in log files:

INFO - Item: {'_id': ObjectId('5c99f8aa5ecb8221dcb3ff73'), 'My object': 'My data'}
INFO - Item: {'_id': ObjectId('5c99ffdf5ecb8221dcb3ff74'), 'My object': 'My data'}
INFO - Item: {'_id': ObjectId('5c99ffe05ecb8221dcb3ff75'), 'My object': 'My data'}

Summary

This is everything that is required - once a connection is created it can be used immediately in Zato services.

Because each connection is actually an instance of the official MongoDB Python client, there are no limits to what can be done with them - building MongoDB-based applications with Zato is now a fully supported possibility.

Zato 3.1 includes new means to manage access to REST services based on input Method and Accept headers in HTTP requests - here is how they can be employed in practice.

A bit of background

Prior to Zato 3.1, one could always build a REST API reacting to individual HTTP verbs by implementing handle_VERB methods in services, such as:

class MyService(Service):

    def handle_GET(self):
        # Reacts to GET requests
        pass

    def handle_POST(self):
        # Reacts to POST requests
        pass

    # Any other handle_VERB method will be used accordingly

This works and will continue to work as expected in all future Zato versions.

However, one aspect of it is that, if one uses SimpleIO, keeping all handler methods in the same service means that all of them share the same SIO definition which is not always desirable - for instance, input to POST may be unrelated to input that DELETE receives.

REST channel URL paths

In Zato 3.1 and newer, it is possible to create REST channels that have the same URL path but different services mounted on each channel, separately for each HTTP verb requires.

That is, whereas previously it was a single service with multiple handle_VERB methods, now it can be a set of services, each reacting to a different HTTP verb and all of them mounted on the same URL path.

In a way, this was supported previously but, if handle_VERB methods were not used, URL paths had to be distinct, e.g.

GET /api/user
DELETE /api/user/delete
POST /api/user/create

In 3.1+, this can be simplified to:

GET /api/user
DELETE /api/user
POST /api/user

Now, each of the combination of verb + path may be unique for a REST channel while previously each channel needed to have its own URL path.

Moreover, because each channel may have its own separate service, it also means that each service may have its own SimpleIO definition and that the service becomes less tied to REST.

HTTP Accept headers

This is a completely new feature in 3.1 which lets one have distinct REST channels depending on the requests's Accept headers.

For instance, let's say that we would like to process incoming invoices under POST /api/invoice but we would like to have two services reacting to the same endpoint, one for JSON and now for PDF invoices.

This can be achieved by configuring HTTP Accept headers in their channels, as below - note that the method and URL path are the same in both cases yet HTTP Accept and services are different because each service reacts to a different value of HTTP Accept:

HTTP Accept header patterns

We can go one better and take advantage of Accept header patterns - with an asterisk meaning any character - this will configure the channel to process requests matching any value that fits in with the pattern, e.g. text/* will mean text/csv, text/xml or anything that starts with text/.

However, seeing as it can be a number of input MIME types, at this point we may need to know what the actual value was - this can be extracted from the WSGI environment via self.wsgi_environ.

Summary

Zato 3.1 saw improvements and backward-compatible changes to how REST channels can be built.

It will now support more use-cases, such as single URL path channels with different HTTP verbs and independent SimpleIO definitions or HTTP Accept headers different for each channel.

In turn, this lets one build REST APIs that are more flexible and elastic in their design to react to different input criteria.

A feature new in Zato 3.1 is the ability to connect to LDAP servers, including Active Directory instances, and this article covers basic administration as programming tasks involved in their usage from Python code.

Creating connections

Connections can be easily created in web-admin. Navigate to Connections -> Outgoing -> LDAP and then click Create a new connection.

The same form works for both regular LDAP and AD - in the latter case, make sure that Auth type is set to NTLM.

The most important information is:

  • User credentials
  • Authentication type
  • Server or servers to connect to

Note that if authentication type is not NTLM, user credentials can be provided using the LDAP syntax, e.g. uid=MyUser,ou=users,o=MyOrganization,dc=example,dc=com.

Right after creating a connection be sure to set its password too - the password asigned by default is a randomly generated one.

Pinging

It is always prudent to ping a newly created connection to ensure that all the information entered was correct.

Note that if you have more than one server in a pool then the first available one of them will be pinged - it is the whole pool that is pinged, not a particular part of it.

Active Directory as a REST service

As the first usage example, let's create a service that will translate JSON queries into LDAP lookups - given username or email the service will basic information about the person's account, such as first and last name.

Note that the conn object returned by client.get() below is capable of running any commands that its underlying Python library offers - in this case we are only using searches but any other operation can also be used, e.g. add or modify as well.

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

# stdlib
from json import loads

# Bunch
from bunch import bunchify

# Zato
from zato.server.service import Service

# Where in the directory we expect to find the user
search_base = 'cn=users, dc=example, dc=com'

# On input, we are looking users up by either username or email
search_filter = '(&(|(uid={user_info})(mail={user_info})))'

# On output, we are interested in username, first name, last name and the person's email
query_attributes = ['uid', 'givenName', 'sn', 'mail']

class ADService(Service):
    """ Looks up users in AD by their username or email.
    """
    class SimpleIO:
        input_required = 'user_info'
        output_optional = 'message', 'username', 'first_name', 'last_name', 'email'
        response_elem = None
        skip_empty_keys = True

    def handle(self):

        # Connection name to use
        conn_name = 'My AD Connection'

        # Get a handle to the connection pool
        with self.out.ldap[conn_name].conn.client() as client:

            # Get a handle to a particular connection
            with client.get() as conn:

                # Build a filter to find a user by
                user_info = self.request.input['user_info']
                user_filter = search_filter.format(user_info=user_info)

                # Returns True if query succeeds and has any information on output
                if conn.search(search_base, user_filter, attributes=query_attributes):

                    # This is where the actual response can be found
                    response = conn.entries

                    # In this case, we expect at most one user matching input criteria
                    entry = response[0]

                    # Convert to JSON so it is easier to handle
                    entry = entry.entry_to_json()

                    # Load from JSON to a Python dict
                    entry = loads(entry)

                    # Convert to a Bunch instance to get dot access to dictionary keys
                    entry = bunchify(entry['attributes'])

                    # Now, actually produce a JSON response. For simplicity's sake,
                    # assume that users have only one of email or other attributes.
                    self.response.payload.message = 'User found'
                    self.response.payload.username = entry.uid[0]
                    self.response.payload.first_name = entry.givenName[0]
                    self.response.payload.last_name = entry.sn[0]
                    self.response.payload.email = entry.mail[0]

                else:
                    # No business response = no such user found
                    self.response.payload.message = 'No such user'

After creating a REST channel, we can invoke the service from command line:

$ curl "localhost:11223/api/get-user?user_info=MyOrganization\\MyUser" ; echo
{
    "message": "User found",
    "username": "MyOrganization\\MyUser",
    "first_name": "First",
    "last_name": "Last",    
    "email": "address@example.com"
}
$

Checking user credentials

A recurrent task in enterprise integrations in checking user credentials on behalf of systems that are not able to connect to AD or LDAP themselves; for instance, because they do not support the LDAP protocol or because a particular architecture disallows for them to make direct connections to backend servers.

To support this use-case, a separate method was added to the Python API specifically to validate user credentials - the code below is everything that is needed to confirm if user credentials are valid:

...

def handle(self):

    # Connection name to use
    conn_name = 'My AD Connection'

    # Credentials to check
    username = 'myuser'
    password = 'mypassword'

    # Get a handle to the connection pool object
    with self.out.ldap[conn_name].conn.client() as client:

        # Check credentials using the pool's configuration
        is_valid = client.check_credentials(username, password)

        if is_valid:
            # Credentials are valid, act accordingly here
            ...

        else:
            # Invalid username or password, return an error here
            ...

Summary

Full support for LDAP and Active Directory connections was added in Zato 3.1 and the Python API exposed grants one access to all the operations possible - offering means to integrate with directories or making them communicate with other technologies or protocols is now just a matter of authoring a service and exposing it through a channel, such as REST or one of the other types that Zato supports.