SimpleIO (SIO)

SIO is a feature of Zato that allows one to develop services in a way that is reusable across multiple data formats and transports. That is, a service can be written once but it’s still possible to expose it, for instance, via XML through AMQP, and via JSON over HTTP.

Once a service has been deployed, no code changes nor restarts are needed to make it available over various access methods.

Note

SIO is not meant to be used when arbitrarily complex requests and responses are needed. You can either return flat objects, such as a bunch of attributes or a list of objects, such a list of groups of attributes. In practice, this covers 90% of use-cases but you need to remember that you can’t return nested structures.

Naturally, when not using SIO, Zato does allow for accepting and returning any documents in JSON, XML or indeed, in any data format.

Note

Zato’s own services - 500+ of them- are exposed through SIO and serve as a good usage example of what SIO is capable of.

Sample usage

An SIO service is one which has an inner class named SimpleIO that conforms to a certain API, for example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        input_required = ('name', 'type')
        output_required = ('is_allowed',)

    def get_data(self):
        if self.request.input.name == 'wendy' and self.request.input.type == 'AXC':
            return True

    def handle(self):
        self.response.payload.is_allowed = self.get_data()

Save it in an sio_example.py file, hot-deploy it and add new channels the service will be exposed over. For this exercise, let's use JSON and SOAP.

JSON channel example

First, let's add a JSON channel:

../_images/sio-example1.png

We can now invoke the service from command-line via curl, notice different output depending on whether conditions get_data looks for are met or not.

$ curl localhost:17010/json/sio-example.my-service -d '{"name":"wendy", "type":"AXC"}'
{"response": {"is_allowed": true}}
$ curl localhost:17010/json/sio-example.my-service -d '{"name":"janet", "type":"AXC"}'
{"response": {"is_allowed": false}}

SOAP channel example

Now, add a SOAP channel - note that the service is the same, you're only using web admin's features here, no code changes are needed, no restarts either.

../_images/sio-example2.png

Here we can use curl too though now we're using SOAP, both requests and responses are more verbose than previously and have been manually spread across several lines to improve clarity.

$ curl localhost:17010/soap/sio-example.my-service \
    -H "SOAPAction:sio-example.my-service" -d '
  <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
      xmlns:zato="https://zato.io/ns/20130518">
    <soapenv:Body>
      <zato:request>
        <zato:name>wendy</zato:name>
        <zato:type>AXC</zato:type>
      </zato:request>
    </soapenv:Body>
  </soapenv:Envelope>'

<?xml version='1.0' encoding='UTF-8'?>
  <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
      xmlns="https://zato.io/ns/20130518">
    <soap:Body>
      <response>
        <zato_env>
          <cid>K41256683804713584730770196331</cid>
          <result>ZATO_OK</result>
        </zato_env>
        <item>
          <is_allowed>true</is_allowed>
        </item>
      </response>
    </soap:Body>
  </soap:Envelope>
$ curl localhost:17010/soap/sio-example.my-service \
    -H "SOAPAction:sio-example.my-service" -d '
  <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
      xmlns:zato="https://zato.io/ns/20130518">
    <soapenv:Body>
      <zato:request>
        <zato:name>janet</zato:name>
        <zato:type>AXC</zato:type>
      </zato:request>
    </soapenv:Body>
  </soapenv:Envelope>'

<?xml version='1.0' encoding='UTF-8'?>
  <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
      xmlns="https://zato.io/ns/20130518">
    <soap:Body>
      <response>
        <zato_env>
          <cid>K14860259454465054570695329428</cid>
          <result>ZATO_OK</result>
        </zato_env>
        <item>
          <is_allowed>false</is_allowed>
        </item>
      </response>
    </soap:Body>
  </soap:Envelope>

Anatomy of an SIO service

SimpleIO attributes

As mentioned above, to use SIO, a service needs to have an inner class called SimpleIO. The class can define a number of attributes.

The example shows a service using all of the attributes SIO allows for:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        request_elem = 'service_request'
        response_elem = 'service_response'
        input_required = ('name', 'type')
        input_optional = ('cust_category', 'priority')
        output_required = ('is_allowed',)
        output_optional = ('session_timeout', 'should_refresh_creds', 'priority')
        default_value = 'UNKNOWN'

    def handle(self):
        self.response.payload.is_allowed = False
        self.response.payload.should_refresh_creds = True
        self.response.payload.priority = self.request.input.priority
$ curl localhost:17010/json/sio-example.my-service -d '{"name":"wendy", "type":"AXC"}'
{"service_response": {"session_timeout": "", "priority": "UNKNOWN",
    "is_allowed": false, "should_refresh_creds": true}}
Attribute Notes Default
request_elem Used with XML only - name of the root element that contains business elements. 'request'
response_elem Name of the response element to wrap payload's elements with. With JSON, it may be set to None in which case no root-level element will be produced in responses at all. 'response'
input_required A tuple of names, each name will have to exist in the request. An exception will be raised if any is missing on input. An empty tuple
input_optional A tuple of names, parameters that can be optionally passed in. No exception will be raised on a missing one. An empty tuple
output_required A tuple of names, the service guarantees that each element will exist in the response. Zato will raise an exception if you declare that a service should return an element and the element isn't returned. An empty tuple
output_optional A tuple of names, they can be returned optionally. Note that when you omit any, Zato will still return that element with an empty value, unless skip_empty_keys is True. An empty tuple
default_value A value to use when an optional element wasn't in the request. None
skip_empty_keys If set to True, empty output optional elements will not be returned. False
force_empty_keys A list of output elements that will be returned even if they are empty and skip_empty_keys is True An empty list

Returning a list of elements

By default, Zato assumes an SIO service returns a flat list of elements, say, details of a customer's product. You need to use the slice notation on the payload object or its .append method if you wish to return a list.

Consider the examples below, they produce the same output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        output_required = ('name', 'type')

    def get_data(self):
        # Imagine a datasource is consulted here
        output = []
        for idx in range(5):
            item = {'name': 'myname-{}'.format(idx), 'type': 'mytype-{}'.format(idx)}
            output.append(item)

        return output

    def handle(self):
        self.response.payload[:] = self.get_data()
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        output_required = ('name', 'type')

    def handle(self):
        for idx in range(5):
            item = {'name': 'myname-{}'.format(idx), 'type': 'mytype-{}'.format(idx)}
            self.response.payload.append(item)
$ curl localhost:17010/json/sio-example.my-service -d '{}'
  {"response": [
     {"type": "mytype-0", "name": "myname-0"},
     {"type": "mytype-1", "name": "myname-1"},
     {"type": "mytype-2", "name": "myname-2"},
     {"type": "mytype-3", "name": "myname-3"},
     {"type": "mytype-4", "name": "myname-4"}
  ]}

How to produce the payload

Zato allows for a range objects to be assigned to the payload which will be able to figure out itself what they are without any need for additional hints from your end. Note that this works equally well whether output is repeated or not.

Assign values directly

Just assign values to the payload:

1
2
3
4
5
6
7
8
9
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        output_required = ('name', 'last_name')

    def handle(self):
        self.response.payload.name = 'Mark'
        self.response.payload.last_name = 'Twain'
$ curl localhost:17010/json/sio-example.my-service -d '{}'
{"response": {"last_name": "Twain", "name": "Mark"}}

Dictionaries

Use a dictionary:

1
2
3
4
5
6
7
8
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        output_required = ('name', 'last_name')

    def handle(self):
        self.response.payload = {'name': 'Mark', 'last_name': 'Twain'}
$ curl localhost:17010/json/sio-example.my-service -d '{}'
{"response": {"last_name": "Twain", "name": "Mark"}}

This also works with dictionary-like objects, such as Bunch is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# Bunch
from bunch import Bunch

# Zato
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        output_required = ('name', 'last_name')

    def get_data(self):

        data = Bunch()
        data.name = 'Robin'
        data.last_name = 'Hood'

        return data

    def handle(self):
        self.response.payload = self.get_data()
$ curl localhost:17010/json/sio-example.my-service -d ''
{"response": {"last_name": "Hood", "name": "Robin"}}

In fact, this will work with a lot of data sources as long as they are able to produce a dictionary-like output.

SQLAlchemy objects

You can directly use SQLAlchemy query results, Zato will convert it itself. This is great if you want to use SQL connections as it simply means you only have to write SQLAlchemy queries without a need for converting their results into the payload format.

Let's say you've defined your database table as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# SQLAlchemy
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class User(Base):
    __tablename__ = 'user'

    id = Column(Integer, primary_key=True)
    name = Column(String(200))
    last_name = Column(String(200))

For the sake of keeping it simple, let's say the table is defined in the same database your ODB resides in. You can simply assign the object returned by SQLAlchemy to the payload and Zato will extract all the attributes out of it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# stdlib
from contextlib import closing

# App's model
from mymodel import User

# Zato
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        input_required = 'id'
        output_required = ('name', 'last_name')

    def handle(self):
        with closing(self.odb.session()) as session:
            self.response.payload = session.query(User).\
                filter(User.id==self.request.input.id).\
                one()
$ curl localhost:17010/json/sio-example.my-service -d '{"id":1}'
{"response": {"last_name": "Freeman", "name": "Django"}}

Datatypes

Zato uses a few conventions when deciding when to convert request and response parameters between various datatypes. This can be helpful because otherwise all the request parameters could've been strings - this is true for both JSON and XML but doubly so for the latter.

Default conventions are:

  • If a parameter's name is 'id' it will be converted to an integer
  • If a parameter ends with '_count', '_id', '_size' or '_timeout' it will be converted to an integer
  • If a parameter begins with 'by_', 'has_', 'is_', 'may_', 'needs_' or 'should_' it will be converted to a bool
  • If a parameter is one of 'auth_data', 'auth_token', 'password', 'password1', 'password2', 'secret_key' or 'token', its value will be automatically encrypted before it is handed over to the service, which means it can be, for instance, stored in the database immediately without a need for manual encryption. Use self.crypto.decrypt to decrypt the value.

All of the defaults are configured through a file called simple-io.conf which each server ships with. Note that new values can be added to it, but existing ones must not be removed because Zato uses them internally. To override any of the defaults, use AsIs datatype, as documented below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
from zato.server.service import Service

class MyService(Service):
    class SimpleIO:
        input_required = ('id', 'customer_id', 'pool_size',
            'job_timeout', 'is_active', 'needs_reset', 'should_continue')

    def handle(self):
        for name, value in self.request.input.items():
            self.logger.info(
              'name:{}, value:{}, type:{}'.format(name, value, type(value)))
$ curl localhost:17010/json/sio-example.my-service -d '{"id":1, "customer_id":3,
  "pool_size":"10", "job_timeout":"300", "is_active":false,
  "needs_reset":"true", "should_continue":false}'
INFO - name:needs_reset, value:True, type:<type 'bool'>
INFO - name:should_continue, value:False, type:<type 'bool'>
INFO - name:customer_id, value:3, type:<type 'int'>
INFO - name:is_active, value:False, type:<type 'bool'>
INFO - name:id, value:1, type:<type 'int'>
INFO - name:job_timeout, value:300, type:<type 'int'>
INFO - name:pool_size, value:10, type:<type 'int'>

It is possible to explicitly specify the type of an input or output parameter. Doing so will will signal to Zato that it shouldn't attempt to perform the default type conversions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
from zato.server.service import AsIs, Boolean, Integer, Unicode, Service

class MyService(Service):
    class SimpleIO:
        input_required = (AsIs('id'), Boolean('reset_required'),
            Integer('type'), Unicode('cust_id'))

    def handle(self):
        for name, value in self.request.input.items():
            self.logger.info(
              'name:{}, value:{}, type:{}'.format(name, repr(value), type(value)))
$ curl localhost:17010/json/sio-example.my-service -d '{"id":"1",
  "reset_required":"true", "type":"30", "cust_id":1}'
INFO - name:cust_id, value:u'1', type:<type 'unicode'>
INFO - name:type, value:30, type:<type 'int'>
INFO - name:reset_required, value:True, type:<type 'bool'>
INFO - name:id, value:u'1', type:<type 'unicode'>
Name Notes
AsIs A pass-through marker, no conversions will be performed even though normally one would have been attempted - it is convenient to use it, for instance, to return identifiers that are not integers. By defaults, field id is understood to represent an integer, but AsIs('id') may hold any value.
Boolean Converted to a bool object
CSV Converted to/from comma-separated values
Date Converts Python's datetime.date objects to string
DateTime Converts Python's datetime.datetime objects to string
Dict Converted to a dictionary
Integer Converted to an integer
List Converted to a list
ListOfDicts Converted to a list of dictionaries
Opaque Similar to AsIs but works with arbitrarily nested structures as well
Unicode Converted to a unicode object

Default values

Each SimpleIO elements may specify a default value that will be used if none is provided in the request, e.g. in the example below self.request.input.is_admin will default to False if is_admin is not given on input.

1
2
3
4
5
6
from zato.server.service import Boolean

class MyService(Service):
    class SimpleIO:
        input_required = 'user_id'
        input_optional = (Boolean('is_admin', default=False), 'user_name', 'user_type')

Similarly, specifying default values in output SimpleIO elements makes it possible to return a value if it is not explicitly provided in the output assigned to self.response.payload:

1
2
3
4
5
from zato.server.service import Int

class MyService(Service):
    class SimpleIO:
        output_optional = Int('max_size', default=100)

If no default is given for an element and default_value is not configured on the SimpleIO's level an empty string will be used as the default value.

Converting date and datetime objects

Elements Date and DateTime may be used to convert, respectively, Python's datetime.date and datetime.datetime objects. This is particularly useful when returning responses based directly on SQLAlchemy objects, such as timestamps or any other dates and times. If data input to the elements is not, accordingly, date or datetime, no conversion will be attempted.

Both elements may optionally receive a parameter representing the format that the resulting string will have. The format must be a value that Python's .strftime can accept, such as '%Y-%m-%d' or '%c'.

DateTime's default format is literal 'iso' which is a shortcut notation indicating that input value should be converted to an ISO-8601 timestamp using its .isoformat() method.

With both Date and DateTime elements, if year is less than 1900, the input format will be ignored and output will be produced using the element's .isoformat() method and in such cases, output dates will be always represented as YYYY-MM-DD.

Element Default format
Date %Y-%m-%d
DateTime iso

Sample usage:

1
2
3
4
5
6
7
class SimpleIO:
  input_required = 'user_id'
  output_required = (Date('join_date'), DateTime('last_seen'))

class SimpleIO:
  input_required = 'user_id'
  output_required = (Date('join_date', '%x'), DateTime('last_seen', '%c'))

Changelog

Version Notes
3.0

Introduced simple-io.conf to keep built-in and custom SimpleIO configuration in

Added new types:

  • Date
  • DateTime

Added new options:

  • skip_empty_keys
  • force_empty_keys
2.0

Added new types:

  • CSV
  • Dict
  • Float
  • List
  • ListOfDicts
  • Opaque
1.0 Added initially