# Structured output

Get JSON out of a model that downstream code can rely on - parsed, validated and retried.

A model asked for JSON returns JSON most of the time - and prose around JSON, markdown fences or a missing field often enough to break anything that assumes otherwise. The service below extracts order data from free-form emails and returns a dictionary that downstream code can rely on, because everything the model returns is parsed, validated and retried before it is returned to any caller.

**Prerequisites.** An [LLM connection](https://zato.io/docs/ai/llm/connections.html) named `Extractor LLM` - any backend from [self-hosted models](https://zato.io/docs/ai/examples/self-hosted-models.html) works - and the skill below.

## The skill {#the-skill}

The skill states the format contract. Keeping the format rules in a [skill](https://zato.io/docs/ai/skills/index.html) means tightening them later is a file edit, not a redeployment - `config/repo/skills/order-extractor/SKILL.md`:

```markdown
---
name: order-extractor
description: Extracts order data from free-form text as strict JSON
---

You extract order data from customer emails.

Respond with a single JSON object and absolutely nothing else - no explanations,
no markdown fences, no text before or after it.

The object has exactly these keys:

* "customer_name" - a string, the customer's name
* "product" - a string, what they are ordering
* "quantity" - an integer, how many they want

When a value is not stated in the text, use null for it.
```

## The service {#the-service}

```python
# -*- coding: utf-8 -*-

# stdlib
import json

# Zato
from zato.server.service import Service

# The keys the model's reply must contain - the same contract the skill states
_required_keys = ('customer_name', 'product', 'quantity')

# How many times a malformed reply is retried before the service gives up
_max_attempts = 3

class ExtractOrder(Service):
    """ Extracts structured order data from free-form text.
    """
    name = 'example.ai.extract-order'

    input = 'text'
    output = 'customer_name', 'product', 'quantity'

    def _parse_reply(self, reply):
        """ Returns the reply as a dict, or None when it is not the JSON we asked for.
        """

        # Models sometimes wrap JSON in markdown fences or prose - taking the slice
        # between the first { and the last } recovers the object from both cases.
        start = reply.find('{')
        end = reply.rfind('}')

        if start == -1 or end == -1:
            return None

        try:
            data = json.loads(reply[start:end + 1])
        except ValueError:
            return None

        # A reply that parses but misses a key breaks callers too
        for key in _required_keys:
            if key not in data:
                return None

        return data

    def handle(self):

        conn = self.llm['Extractor LLM']
        text = self.request.input.text

        for attempt in range(_max_attempts):

            response = conn.invoke(text, skill='order-extractor')
            data = self._parse_reply(response['text'])

            if data is not None:
                self.response.payload = data
                return

            self.logger.info('Malformed extractor reply, attempt %d: %s', attempt + 1, response['text'])

        raise Exception(f'No valid JSON after {_max_attempts} attempts')
```

## Run the service {#run-the-service}

```bash
curl localhost:11223/example/extract-order -d '{"text":"Hi, this is Jane Smith, I would like to order two ergonomic keyboards."}'
```

Expected output:

```json
{"customer_name": "Jane Smith", "product": "ergonomic keyboard", "quantity": 2}
```

The REST channel:

```yaml
channel_rest:
  - name: example.ai.extract-order
    service: example.ai.extract-order
    url_path: /example/extract-order
```

## Failure behavior {#failure-behavior}

The retry loop is the failure path - a reply wrapped in prose or missing a key is logged with its attempt number and asked again, and the caller receives either a valid object or the final exception. Each attempt is a separate call in the [audit log](https://zato.io/docs/ai/llm/connections.html#the-audit-trail), so an extraction that routinely needs two attempts is visible there as double traffic - a measured signal that the skill's wording needs tightening or the model needs changing. Each retry's cost is in the response's usage dictionary - see [token usage](https://zato.io/docs/ai/examples/token-usage.html).

## See also {#see-also}

- [Skills](https://zato.io/docs/ai/skills/index.html) - The instruction files that spell out the format contract
- [Token usage](https://zato.io/docs/ai/examples/token-usage.html) - What each attempt of the retry loop costs
- [Evaluating AI flows](https://zato.io/docs/ai/examples/evaluating-ai-flows.html) - Structural assertions over responses like this one