Command-line tools

Call command-line tools from services as named connections.

Use a command-line tool as a connection type when a system ships a binary instead of a client library. With the Connector SDK, a tool takes one of two shapes:

One-shot commands: the connector runs the binary per call and returns its output - these become plain client methods
Long-lived tools: the tool runs for the connection's lifetime and calls read from it - these become supervised helper processes started with start_process

For example, ngrok is a long-lived tool. The tunnel starts with the connection, the connector reads the tunnel's address from ngrok's local API and exposes it as a method, and deleting the definition stops the tunnel.

The connector module

One connector can cover both shapes. The following module runs a tunnel session as a supervised helper process and wraps a one-shot status command as a plain method:

# -*- coding: utf-8 -*-

# stdlib
import socket
import subprocess
import time

# Zato
from zato.common.sdk import Connector, Field

# How long to wait for the CLI tool to start serving its local API, in seconds.
_startup_timeout = 15

# How long a one-shot command may take, in seconds.
_command_timeout = 30

class TunnelClient:
    """ A client for the CLI tool's local API - one socket connection per request, the interface
    tools like ngrok provide for reading tunnel details.
    """
    def __init__(self, host:'str', port:'int') -> 'None':
        self.host = host
        self.port = port

    def send(self, data:'str') -> 'str':

        with socket.create_connection((self.host, self.port)) as conn:
            conn.sendall(f'{data}\n'.encode('utf8'))

            with conn.makefile('r', encoding='utf8') as reader:
                response = reader.readline()

        return response.strip()

class TunnelConnector(Connector):
    """ Wraps a CLI tool as a connection type. The long-lived session runs as a supervised helper
    process for as long as the connection runs, and one-shot commands are plain methods that run
    the binary and return its output.
    """
    type = 'tunnel'

    # Configuration schema
    binary_path = Field.Text()

    def create_client(self) -> 'TunnelClient':

        # The long-lived session starts with the connection and stops with it - deleting
        # the definition stops the tunnel.
        process = self.start_process([self.config.binary_path, 'serve', '{port}'])

        client = TunnelClient('127.0.0.1', process.port)

        # Wait until the tool's local API accepts connections.
        deadline = time.monotonic() + _startup_timeout

        while True:
            try:
                client.send('address')
            except OSError:
                if time.monotonic() > deadline:
                    raise Exception(f'The tunnel did not start within {_startup_timeout}s')
                time.sleep(0.2)
            else:
                break

        return client

    def ping(self, client:'TunnelClient') -> 'None':
        client.send('address')

    def get_address(self) -> 'str':
        """ The tunnel's address, read from the tool's local API.
        """
        return self.client.send('address')

    def get_status(self, name:'str') -> 'str':
        """ A one-shot command wrapped as a client method - run the binary, return its output.
        """
        result = subprocess.run(
            [self.config.binary_path, 'status', name],
            capture_output=True, text=True, timeout=_command_timeout, check=True)

        return result.stdout.strip()

The platform supervises the long-lived session. When the tool crashes, the platform rebuilds the connection, which starts the tool again.

One-shot commands require nothing beyond subprocess.run inside a plain method, and each such method receives the same watchdog timeout as every invocation.

With the real ngrok, create_client runs ngrok http 8080 and get_address reads the public URL from ngrok's local API - the structure stays the same.

Create a definition

Create the definition in an enmasse file, under the custom_tunnel key - custom_ followed by the connector's type:

custom_tunnel:
  - name: My Tunnel
    binary_path: /usr/local/bin/ngrok

The tunnel runs for as long as its definition exists and services read its address through the named connection.

Page	What it covers
SDK reference	Field types, lifecycle methods and the behavior of every invocation
Java, .NET and native code	Clients that run as supervised helper processes next to the server
Connector SDK tutorial	Build a complete connector and call it from a service

Command-line tools

The connector module

Create a definition

See also