# Calling LLMs from clinical services

Invoke a model from a clinical service in one line of Python.

[Video: New LLM connection](https://zatosource-production.b-cdn.net/docs/gfx/outgoing/llm-create.webm?v=1790440805)

Dashboard menu: AI > LLM connections

This page builds a service that generates a plain-language ward notification from an incoming admission event.

The service calls the model through a named LLM connection. The connection stores the provider protocol, model, address, credentials and runtime limits outside the service code, and it determines where the prompt is sent.

## Before you begin {#before-you-begin}

You need:

- A running Zato environment. The [Docker installation guide](https://zato.io/docs/admin/guide/install/docker.html) sets one up.
- A [REST channel](https://zato.io/docs/dev/rest/index.html) that invokes your service.
- An LLM connection named `Clinical LLM`, created under [Create the connection](https://zato.io/docs/dev/healthcare/ai/llm.html#create-the-connection) below.

## Call a model from a service {#call-a-model-from-a-service}

The REST channel invokes the service with the admission event. The service sends the selected name, patient class, location and event time to the configured model:

```python
# -*- coding: utf-8 -*-

# Zato
from zato.server.service import Service

# #####################################################################
# #####################################################################

class WardNotification(Service):
    """ Generates a ward notification from an admission.
    """
    name = 'clinical.ward-notification'

    input = 'family_name', 'patient_class', 'location', 'event_time'

    def handle(self) -> 'None':

        # Read the admission event received by the REST channel ..
        data = self.request.input

        # .. select the fields needed by the notification ..
        family_name   = data.family_name
        patient_class = data.patient_class
        location      = data.location
        event_time    = data.event_time

        # .. build the prompt sent to the model ..
        prompt = (
            'Write a one-sentence ward notification for this admission: '
            f'patient {family_name}, class {patient_class}, '
            f'location {location}, at {event_time}.'
        )

        # .. invoke the named connection ..
        connection = self.llm['Clinical LLM']
        response = connection.invoke(prompt)

        # .. and log the returned notification.
        notification = response['text']
        self.logger.info('Notification: %s', notification)
```

Invoke the service through its REST channel:

```text
curl http://localhost:11223/ward-notification -d '{"family_name": "Smith", "patient_class": "inpatient", "location": "W3, room 12", "event_time": "2026-03-15 12:00"}'
```

The logged notification reads:

```text
Notification: Smith was admitted as an inpatient to W3, room 12, at 12:00 on 15 March 2026.
```

The model receives the selected fields rather than the complete admission event. The patient's name and location remain PHI, so the connection determines whether they are sent to a hosted provider or a model operated in the selected environment. The `invoke` call, its response dictionary and its error behavior are covered under [invoking LLMs from services](https://zato.io/docs/ai/llm/invoke.html), and conversations that keep their history use [chat](https://zato.io/docs/ai/llm/chat.html) instead.

## Create the connection {#create-the-connection}

In the Dashboard, open **AI** > **LLM connections** and create a connection. Services select it through `self.llm[name]`. Its model setting determines the provider protocol, while the address and [API key](https://zato.io/docs/security/api-keys.html) determine the destination and authentication. Pool size, timeout, token limits and history limits are documented in the [connections reference](https://zato.io/docs/ai/llm/connections.html), and the connection can also live as enmasse YAML in git, imported per environment as [GitOps](https://zato.io/docs/ai/enmasse.html) describes.

## What is recorded and what it costs {#what-is-recorded-and-what-it-costs}

Every call through the connection is recorded with its outcome, duration and token usage, for successful and failed calls alike, with the provider's own error kept verbatim, as the [audit trail](https://zato.io/docs/ai/llm/connections.html#the-audit-trail) describes.

Every response also includes a `usage` dictionary with `input_tokens` and `output_tokens`, using the same two names regardless of the provider:

```python
# Read the usage reported with the response ..
usage = response['usage']

# .. extract the provider-independent token counts ..
input_tokens  = usage['input_tokens']
output_tokens = usage['output_tokens']

# .. and record them.
self.logger.info('Tokens: %s in, %s out', input_tokens, output_tokens)
```

For more details, the [token usage example](https://zato.io/docs/ai/examples/token-usage.html) computes the cost per caller from the dictionary, and [cost and limits](https://zato.io/docs/ai/cost-and-limits.html) documents the available spending limits.

- [Expose services as MCP](https://zato.io/docs/dev/healthcare/ai/agents.html)
- [Interface health for AI agents](https://zato.io/docs/dev/healthcare/ai/interface-health.html)