Handshake protocols and pooled connections

Logon handshakes and one-call-at-a-time sessions, pooled by the platform.

Some systems require a logon before the first request. The system assigns a session to the logon and every later call runs within that session - one call at a time, in order. Mainframe gateways, trading systems and many older banking protocols work this way.

Concurrent calls therefore each need a session of their own, so the Connector SDK pools the connections. You write the connector as if there were a single connection - the platform owns the pool, builds connections as the load requires them and each call borrows one for its whole duration.

The connector module

The connector wraps a mainframe gateway that expects a logon line first and answers it with a session ID. It subclasses PooledConnector instead of Connector, and the platform then owns a pool of connections for each definition.

# -*- coding: utf-8 -*-

# stdlib
import socket

# Zato
from zato.common.sdk import ConnectionLost, Field, PooledConnector

class MainframeConnection:
    """ One connection to a mainframe gateway. The gateway requires a logon handshake first and
    then serves one call at a time per session, which is why the platform pools these connections.
    """
    def __init__(self, host:'str', port:'int', logon_token:'str') -> 'None':

        # Connect and keep the socket open for as long as the connection exists ..
        self.socket = socket.create_connection((host, port))
        self.reader = self.socket.makefile('r', encoding='utf8')

        # .. log on first - the gateway answers with the session it assigned ..
        self._send_line(f'logon {logon_token}')
        response = self.reader.readline().strip()

        # .. anything other than an ok reply means the logon was rejected.
        if not response.startswith('ok '):
            raise Exception(f'Logon failed -> {response}')

        self.session_id = response[len('ok '):]

    def _send_line(self, data:'str') -> 'None':
        self.socket.sendall(f'{data}\n'.encode('utf8'))

    def send(self, data:'str') -> 'str':
        self._send_line(data)
        return self.reader.readline().strip()

    def close(self) -> 'None':
        self.reader.close()
        self.socket.close()

class MainframeConnector(PooledConnector):
    """ Wraps the mainframe gateway as a connection type that services access through self.out.mainframe.
    """
    type = 'mainframe'

    # Configuration schema
    host = Field.Text()
    port = Field.Int(default=9960)
    logon_token = Field.Secret()

    def create_client(self) -> 'MainframeConnection':
        conn = MainframeConnection(self.config.host, self.config.port, self.config.logon_token)
        self.logger.info('Mainframe session `%s` logged on for `%s`', conn.session_id, self.name)
        return conn

    def ping(self, conn:'MainframeConnection') -> 'None':

        # A closed socket produces an empty line - raising ConnectionLost makes the platform
        # discard the connection.
        response = conn.send('ping')
        if not response.endswith('ping'):
            raise ConnectionLost(f'The gateway did not answer a ping -> {response!r}')

    def on_stop(self, conn:'MainframeConnection') -> 'None':
        conn.close()
        self.logger.info('Mainframe session `%s` closed for `%s`', conn.session_id, self.name)

    def send_command(self, command:'str') -> 'str':

        # Each call borrows one connection for its whole duration - the gateway serves
        # one call at a time per session.
        with self.get_connection() as conn:
            return conn.send(command)

create_client builds one connection each time the pool grows, up to the pool's size.

Invocation methods reach the remote end through get_connection. It borrows a connection from the pool for the duration of the with block and the borrowing call has the connection to itself.

ping, on_stop and the other lifecycle methods receive one pooled connection, the same way create_client produced it.

The logon handshake runs in the connection's __init__, so every connection enters the pool already logged on.

Pool hooks

Two optional hooks run around every borrow. Use them when connections accumulate conversational state that has to be reset between callers:

    def on_get_from_pool(self, conn:'MainframeConnection') -> 'None':
        # Runs each time a call borrows this connection - reset conversational state here
        ...

    def on_return_to_pool(self, conn:'MainframeConnection') -> 'None':
        # Runs each time a with block ends - clean up before the next caller borrows the connection
        ...

Both hooks are optional - declare them when your protocol requires a reset between callers.

Create a definition

Create the definition in an enmasse file, under the custom_mainframe key - custom_ followed by the connector's type:

custom_mainframe:
  - name: My Mainframe
    host: 10.152.81.20
    port: 9960
    logon_token: my-logon-token
    pool_size: 3

pool_size caps how many connections the pool holds. Connections are built lazily - a definition without concurrent load keeps the single connection its first ping created.

Call the connection from a service

Services call the connector's methods like any other connection - the pool stays inside the connector:

# -*- coding: utf-8 -*-

# Zato
from zato.server.service import Service

class SendMainframeCommand(Service):
    """ Sends one command to the mainframe gateway over a pooled connection.
    """
    name = 'demo.mainframe.send-command'

    input = 'command'

    def handle(self) -> 'None':
        conn = self.out.mainframe['My Mainframe']
        response = conn.send_command(self.request.input.command)
        self.response.payload = response

When several invocations run concurrently, each is served by a distinct connection with its own session. A call waits only when every connection in the pool is busy.

See also

PageWhat it covers
SDK referenceField types, lifecycle methods and the behavior of every invocation
Multiplexed requestsThe opposite model - many concurrent requests over one shared socket
Connector SDK tutorialBuild a complete connector and call it from a service