# Python libraries in a separate interpreter

Run a library in the interpreter it requires and call it from services as if it were local.

Run a Python library in an interpreter of its own when it requires a different Python version, when it conflicts with the server's dependencies or when it misbehaves in a shared process. The [Connector SDK](https://zato.io/docs/sdk/index.html) runs the library in the interpreter you point it at - any Python on the machine - and your connector calls it as if it were local.

The hosting side ships with Zato - a stock runner module that any interpreter runs by path - and the module you write for it is plain Python.

## The module with your library {#the-module-with-your-library}

The module contains plain module-level functions and imports only the library it wraps:

```python
# -*- coding: utf-8 -*-

# Anything the library needs
import the_library_that_cannot_run_in_the_server

def ping():
    return 'pong'

def transform(text):
    return the_library_that_cannot_run_in_the_server.process(text)
```

## The connector module {#the-connector-module}

The connector starts the library's interpreter with `start_process` and calls it through `RunnerClient` - each call names a function of your module:

```python
# -*- coding: utf-8 -*-

# stdlib
import time

# Zato
from zato.common.sdk import Connector, Field
from zato.common.sdk import runner
from zato.common.sdk.runner import RunnerClient

# How long to wait for the runner process to start accepting connections, in seconds.
_startup_timeout = 15

class TextProcConnector(Connector):
    """ Wraps a Python library that runs in a separate interpreter - the stock runner hosts
    the library's module and this connector calls it over a local socket.
    """
    type = 'textproc'

    # Configuration schema - which interpreter runs the runner and which module the runner exposes.
    python_path = Field.Text()
    module_path = Field.Text()

    def create_client(self) -> 'RunnerClient':

        # Run the stock runner as a supervised helper process, in a clean interpreter -
        # the runner depends on the standard library only, so any interpreter can run it by path.
        command = [self.config.python_path, runner.__file__, '{port}', self.config.module_path]
        process = self.start_process(command)

        client = RunnerClient('127.0.0.1', process.port)

        # Wait until the runner accepts connections.
        deadline = time.monotonic() + _startup_timeout

        while True:
            try:
                client.call('ping')
            except OSError:
                if time.monotonic() > deadline:
                    raise Exception(f'The runner did not start within {_startup_timeout}s')
                time.sleep(0.2)
            else:
                break

        return client

    def ping(self, client:'RunnerClient') -> 'None':
        client.call('ping')

    def transform(self, text:'str') -> 'str':
        return self.client.call('transform', text)
```

`python_path` selects the interpreter that runs the library - any Python on the machine. The runner depends only on the standard library, so a bare interpreter is enough.

The platform supervises the library's process like any helper started with `start_process`. When the process exits, the platform rebuilds the connection, which starts a new one.

`RunnerClient.call` sends one JSON object per line and reads one back. An exception raised by your module arrives on the connector's side as an exception.

## Create a definition {#create-a-definition}

Create the definition in an [enmasse](https://zato.io/docs/admin/enmasse.html) file, under the `custom_textproc` key - `custom_` followed by the connector's type:

```yaml
custom_textproc:
  - name: My TextProc
    python_path: /opt/python312/bin/python
    module_path: /opt/company/textproc_module.py
```

The library runs in the interpreter the definition names and services call it through the named connection.

## See also {#see-also}

- [SDK reference](https://zato.io/docs/sdk/reference.html) - Field types, lifecycle methods and the behavior of every invocation
- [Java, .NET and native code](https://zato.io/docs/sdk/foreign-runtimes.html) - Vendor components run as supervised helper processes
- [Connector SDK tutorial](https://zato.io/docs/sdk/tutorial.html) - Build a complete connector and call it from a service