This blog post discusses an integration scenario that showcases a new feature in Zato 3.0 - SMS texting with Twilio.

Use-case

Suppose you'd like to send text messages that originate from multiple sources, from multiple systems communicating natively over different protocols, such as the most commonly used ones:

  • REST
  • AMQP
  • WebSockets
  • FTP
  • WebSphere MQ

Naturally, the list could grow but the main points are that:

  • Ubiquitous as it is, HTTP is far from being the only protocol used in more complex environments
  • You don't want to distribute credentials to Twilio to each of backend or frontend systems that wants to text

The solution is to route all the messages through a dedicated Zato service that will:

  • Offer to each system communication in their own native protocol
  • Be the only place where credentials are kept

Let's say that the system that we are building will send text messages informing customers about the availability of their order. For simplicity, only REST and AMQP will be shown below but the same principle will hold for other protocols that Zato supports.

Screenshots

Code

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

# Zato
from zato.server.service import Service

class SMSAdapter(Service):
    """ Sends template-based text messages to users given on input.
    """
    name = 'sms.adapter'

    class SimpleIO:
        input_required = ('user_name', 'order_no')

    def get_phone_number(self, user_name):
        """ Returns a phone number by user_name.+1234567890
        In practice, this would be read from a database or cache.
        """
        users = {
            'mary.major': '+15550101',
            'john.doe': '+15550102',
        }

        return users[user_name]

    def handle(self):

        # In a real system there would be more templates,
        # perhaps in multiple natural languages, and they would be stored in files
        # on disk instead of directly in the body of a service.
        template = "Hello, we are happy to let you know that" \
            "your order #{order_no} is ready for pickup."

        # Get phone number from DB
        phone_number = self.get_phone_number(self.request.input.user_name)

        # Convert the template to an actual message
        msg = template.format(order_no=self.request.input.order_no)

        # Get connection to Twilio
        sms = self.out.sms.twilio.get('My SMS')

        # Send messages
        sms.conn.send(msg, to=phone_number)

In reality, the code would contain more logic, for instance to look up users in an SQL or Cassandra database or to send messages based on different templates but to illustrate the point, the service above will suffice.

Note that the service uses SimpleIO which means it can be used with both JSON and XML even if only the former is used in the example.

This is the only piece of code needed and the rest is simply configuration in web-admin described below.

Channel configuration

The service needs to be mounted on channels - in this scenario it will be HTTP/REST and AMQP ones but could be any other as required in a given integration project.

In all cases, however, no changes to the code are needed in order to support additional protocols - assigning a service to a channel is merely a matter of additional configuration without any coding.

Screenshots

Screenshots

Screenshots

Screenshots

Twilio configuration

Fill out the form in Connections -> SMS -> Twilio to get a new connection to Twilio SMS messaging facilities. Account SID and token are the same values that you are given by Twilio for your account.

Default from is useful if you typically send messages from the same number or nickname. On the other hand, default to is handy if the recipient is usually the same for all messages sent. Both of these values can be always overridden on a per call basis.

Screenshots

Screenshots

Invocation samples

We can now invoke the service from both curl and RabbitMQ's GUI:

Screenshots

Screenshots

Result

In either case, the result is a text message delivered to the intended recipient :-)

Screenshots

There is more!

It is frequently very convenient to test connections without actually having to develop any code - this is why SMS Twilio connections offer a form to do exactly that. Just click on 'Send a message', fill in your message, click Submit and you're done!

Screenshots

Screenshots

Summary

Authoring API services, including ones that send text messages with Twilio is an easy matter with Zato.

Multiple input protocols are supported out of the box and you can rest assured that API keys and other credentials never sprawl all throughout the infrastructure, everything is contained in a single place.

Zato 2.0.8 has just been released - check here for installation instructions under Ubuntu, RHEL/CentOS, Debian, Docker and more.

This is primarily a regular patch release that closes GitHub tickets opened since 2.0.7 was published.

But there is more - several performance improvements have been backported from the upcoming 3.0 release and now your API servers will be up to 30% faster merely by upgrading from 2.0.7 to 2.0.8.

Zato is an open-source Python-based integration platform for ESB, SOA, REST, APIs and Cloud Integrations. Zato focuses on APIs, middleware and backend systems exclusively.

If you're thinking of building distributed systems with Python and need to handle business scenarios or processes of any complexity, from simple to large, Zato is the platform to choose.

Here are some screenshots to whet your appetite - head over to the documentation index for more!

Screenshots

Screenshots

Screenshots

Screenshots

Screenshots

Screenshots

Screenshots

Screenshots

Screenshots

Screenshots

Screenshots

Zato 3.0 will sport a completely rewritten AMQP subsystem and the changes are nothing short of exciting. Put simply, everything is faster, leaner and even more robust.

Synchronous channels

AMQP channels are now synchronous which means that an AMQP broker will wait for a Zato service to confirm that the message was received or it was not. In earlier Zato versions, AMQP channels worked in an implicit always-acknowledge mode. Now each message taken off an AMQP queue can be explicitly acknowledged or rejected, as below:

from zato.server.service import Service

class MyService(Service):

    def handle(self):
        if 'foo' in self.request.payload:
            self.request.amqp.ack()
        else:
            self.request.amqp.reject()

Message details

Channels have access to all metadata regarding an incoming message, for instance:

from zato.server.service import Service

class MyService(Service):

    def handle(self):
        self.logger.info(self.request.amqp.msg)

Screenshots

Connection pools

Previously, a single connection pool was shared by all channels and outgoing connections to a single AMQP broker.

Now each channel or outgoing connection may have its own pool, managed independently, which lets Zato servers use multiple CPUs in AMQP connections to improve performance - the underlying framework for connection pools was redesigned to achieve drastically reduced RAM usage - from megabytes down to kilobytes for each pool.

Channels:

Screenshots

Outgoing connections:

Screenshots

Correlating connections and consumers

When using AMQP GUIs, it was already possible to easily find Zato connections but the idea has been extended and now each connection, channel and consumer carry accounting information on which Zato component is the source or consumer of a message, including remote hostname, process ID, thread/greenlet ID and consumer tag prefix:

Screenshots

Screenshots

Get it from GitHub

A preview version of Zato 3.0 can be already installed from source code - all the new AMQP features are in the default branch, called 'main'. And remember to always feel free to get in touch - leave feedback in the forum or contact support if you need anything!

This is a preview of an upcoming feature in Zato 3.0 that will let one generate and distribute specifications for API services.

Introduction

Let's consider the following two modules with three services related to customers and customer cases. The implementation is left out so as to focus on I/O only.

# api1.py

from zato.server.service import Service

namespace = 'my.api.customer'

class GetCustomer(Service):
    """ Returns basic information about a customer.
    """
    name = 'get-customer'

    class SimpleIO:
        input_required = ('cust_id',)
        input_optional = ('cust_type', 'segment',)
        output_required = ('name', 'is_active', 'region')
        output_optional = ('last_seen',)

    def handle(self):
        pass # Skip implementation

class UpdateCustomer(Service):
    """ Updates basic information about a customer.
    """
    name = 'update-customer'

    class SimpleIO:
        input_required = ('name', 'is_active', 'region')
        output_required = ('result',)

    def handle(self):
        pass # Skip implementation
# api2.py

from zato.server.service import Service

namespace = 'my.api.customer-cases'

class GetCustomerCase(Service):
    """ Returns basic information about an individual case opened by a customer.
    """
    name = 'get-customer-cases'

    class SimpleIO:
        input_required = ('case_id',)
        output_required = ('cust_id', 'is_open', 'status', 'last_contact_date',)
        output_optional = ('supervisor_id', 'region_id')

    def handle(self):
        pass # Skip implementation

Screenshots

In Zato 3.0, and already available if installed from source, a specification for such services will look like below:

Screenshot

Screenshot

Screenshot

Screenshot

Key points

  • Services can be grouped into namespaces
  • For each service its documentation, as extracted from docstrings, is provided
  • Each service can optionally export information what other services it invokes, i.e. depends on
  • Input/Output for each service is based on SimpleIO
  • All parameters are automatically converted to correct types, e.g. is_active is a boolean, cust_id an integer
  • Brand-specific information and CSS can be applied to customize the output - here 'My API spec' is only a generic name that can be replaced with anything else
  • The API specification is output using a service and REST channels thus it can be secured as required just like any other channel in Zato

Future plans

While for now the output is the documentation, the underlying backend is already capable to automatically generate data fpr other API specification formats such as Swagger/OpenAPI, RAML or WSDL - this is exactly what the next Zato version will do, it will generate specs in several formats for easy consumption by API clients.

Overview

While JSON or SOAP-wrapped messages are ubiquitous in HTTP-based APIs, there are times when unformatted messages are of advantage.

This article will explain how to create Zato endpoints accessing such data and to top it off, will also introduce the concept of outgoing FTP connections through which requests can be stored at remote FTP resources.

Code

Hot-deploy the following service onto a Zato cluster.

The key point is self.request.raw_request - this is how a programmer can access input data as it was received by Zato prior to its de-serialization from JSON or XML.

Note that this attribute will be available even if parsing took place, that is, one can always access raw request no matter what data format was used on input.

# Zato
from zato.common.util import fs_safe_now
from zato.server.service import Service

class FTPHandler(Service):
    """ Stores anything that is given on input at a remote FTP server
    pointed to by an outgoing connection called 'my.ftp'.
    """
    def handle(self):

        # Obtain handle to an FTP connection by its name
        conn = self.outgoing.ftp.get('my.ftp')

        # Create file name in a format that is safe for file-systems,
        # i.e. no colons or whitespace.
        file_name = '/home/api/request.{}'.format(fs_safe_now())

        # Store file on FTP
        conn.setcontents(file_name, self.request.raw_request)

Configuration

Now that the code is deployed, we need configuration to plug it into the whole framework. Use web-admin to create two objects per screenshots below.

One is an HTTP channel that exposes the deployed service through an HTTP channel - note that 'Data format' was left blank so as not to require JSON or XML on input.

Screenshot

The other is a definition of the FTP server that the service will store files in.

Screenshot

Don't forget to set password for the newly configured FTP connection to log in with in run-time.

Screenshot

As always with Zato, one's code never accesses any resources directly, everything is routed through named objects such as self.outgoing.ftp['my.ftp'] that actually point to a required resource. In this manner, if anything needs to be reconfigured, it can be done on fly, without changes to code and without server restarts.

Testing

With code deployed and configuration created, we can now upload sample data from command line and confirm it's made its way to an FTP server.

Screenshot

And sure enough - the file is there!

Screenshot

Screenshot

Summary

Accessing raw requests in Zato services is trivially easy. Likewise, making use of FTP connections is a matter of a single line of code, while hot-deployment and configuration mean that changes introduced in web-admin are immediately reflected all throughout a Zato cluster.

Want more?