Understanding Zato server startup callable objects

2020-06-23, by Dariusz Suchojad

Zato startup callable objects are a means through which arbitrary Python functions or classes can be invoked when a server is booting up in order to influence its configuration or setup, even before any service is invoked.

This technique offers customization possibilities in addition to the standard configuration options set before a server starts - read on to learn details.

Startup life-cycle

When a Zato server is starting, it goes through several stages. Most of them are internal details but two of them are exposed to users. Their names are FS_CONFIG_ONLY and AFTER_STARTED.

• FS_CONFIG_ONLY - one of the earliest stages during which only server configuration files exist.

  No connections to any resources are opened yet, no services are deployed, the server does even have a connection to its configuration databases established - the only piece of configuration in existence are server config files.

  During this stage it is possible to make broad, low-level changes to configuration, such as reading configuration options from external endpoints, as in the first example below.

• AFTER_STARTED - one of the latest stages in the process during which all of the configuration is already read in.

  All services are deployed and most HTTP-based connections are started. Notably, connections started in connectors, e.g. Odoo or WebSockets may be not started yet.

  This stage is useful if access to services is needed, as in the second example which invokes an internal service to create a new cache definition.

How to configure startup callable objects

A startup callable is a regular Python function or a Python class placed on a server's PYTHONPATH. For instance, if Zato is installed to /opt/zato/current, placing a Python module my_startup.py in, depending on one's Zato version, /opt/zato/current/extlib or /opt/zato/current/zato_extra_paths will make it appear on PYTHONPATH.

Next, add a dotted name of the callable to server.conf, under the [misc] stanza. For instance, if a function called "update_fs_config" is in a module named "my_startup.py", the configuration will look like below:

[misc]
...
startup_callable=my_startup.create_cache
...

Note that key "startup_callable" can be a comma-separated list of callable objects to invoke, such as:

[misc]
...
startup_callable=my_startup.update_fs_config, my_startup.create_cache
...

Each callable will be invoked in the order defined in the key and each will need to decide whether it is interested in a particular phase or not.

Python code

The only part left now is to create a callable - this can be anything in Python that can be invoked and the example below uses two functions, already registered in server.conf above.

The code consists of the usual imports, including ones for type completion hints, followed by actual callable functions.

Note that each callable is given a PhaseCtx object on input - this is an object that describes the current stage, the current phase the server is going through.

A callable will filter out stages it is not interested, in this case, each callable is interested in a single one only but there is nothing preventing a callable to handle more than one.

Note that there are no limits to what a callable can do - it has access to all the Python libraries that a server has on PYTHONPATH, and, depending on the stage, to all the services already deployed, all of which means that there are virtually no limits to customization choices of the startup process.

# -*- coding: utf-8 -*-

# Requests
import requests

# Zato
from zato.common import CACHE, SERVER_STARTUP
from zato.common.util import as_bool

# Type completion imports
if 0:
    from zato.server.base.parallel import ParallelServer
    from zato.server.startup_callable import PhaseCtx

    ParallelServer = ParallelServer
    PhaseCtx = PhaseCtx

def update_fs_config(ctx):
    """ This callable has access to only to file-system based configuration files.
    """
    # type: (PhaseCtx)

    if ctx.phase == SERVER_STARTUP.PHASE.FS_CONFIG_ONLY:

        # server.conf
        server_config = ctx.kwargs['server_config']

        # pickup.conf
        pickup_config = ctx.kwargs['pickup_config']

        # simple-io.conf
        sio_config = ctx.kwargs['sio_config']

        # sso.conf
        sso_config = ctx.kwargs['sso_config']

        # Base directory the server was installed to
        base_dir = ctx.kwargs['base_dir']

        # Consult a remote resource and check
        # if Cassandra connections should be enabled
        response = requests.get('https://example.com/')

        # Convert to a bool object
        is_enabled = as_bool(response.text)

        # Update the server configuration in place
        server_config.component_enabled.cassandara = is_enabled

def create_cache(ctx):
    """ This is the callable that is invoked when the server is started.
    """
    # type: (PhaseCtx)

    if ctx.phase == SERVER_STARTUP.PHASE.AFTER_STARTED:

        # Server object where all our services are deployed to already
        server = ctx.kwargs['server'] # type: ParallelServer

        # Service to invoke
        service = 'zato.cache.builtin.create'

        # Input data to the service
        request = {
            'cluster_id': server.cluster_id,
            'name': 'my.cache',
            'is_active': True,
            'is_default': False,
            'max_size': 10000,
            'max_item_size': 10000,
            'extend_expiry_on_get': True,
            'extend_expiry_on_set': True,
            'sync_method': CACHE.SYNC_METHOD.IN_BACKGROUND.id,
            'persistent_storage': CACHE.PERSISTENT_STORAGE.NO_PERSISTENT_STORAGE.id,
            'cache_type': CACHE.TYPE.BUILTIN,
        }

        # Create a cache object by invoke the service
        server.invoke(service, request)