# Token usage

Convert per-call token counts to cost per caller, logged in a form that sums.

Every response includes what the call consumed - the `usage` dictionary described under [invoke](https://zato.io/docs/ai/llm/invoke.html). The service below derives cost per department from those numbers, logged per call in a form that sums with standard tools.

**Prerequisites.** An [LLM connection](https://zato.io/docs/ai/llm/connections.html) named `Shared LLM` - the one connection several departments share is the typical case in which per-caller cost matters.

## The service {#the-service}

```python
# -*- coding: utf-8 -*-

# Zato
from zato.server.service import Service

# The provider's price per one million tokens, in USD - from its price list
_price_per_million_input = 0.15
_price_per_million_output = 0.60

class AskWithAccounting(Service):
    """ Answers a question and logs what it cost, attributed to the caller.
    """
    name = 'example.ai.ask-with-accounting'

    input = 'department', 'question'
    output = 'answer', 'cost_usd'

    def handle(self):

        conn = self.llm['Shared LLM']

        response = conn.invoke(self.request.input.question)

        # The two counts are always present, regardless of the model that answered
        input_tokens = response['usage']['input_tokens']
        output_tokens = response['usage']['output_tokens']

        input_cost = input_tokens * _price_per_million_input / 1_000_000
        output_cost = output_tokens * _price_per_million_output / 1_000_000
        cost_usd = round(input_cost + output_cost, 6)

        # One parseable line per call - key=value, so any log tooling can sum it
        self.logger.info('llm-cost department=%s input_tokens=%d output_tokens=%d cost_usd=%f',
            self.request.input.department, input_tokens, output_tokens, cost_usd)

        self.response.payload.answer = response['text']
        self.response.payload.cost_usd = cost_usd
```

## Run the service {#run-the-service}

```bash
curl localhost:11223/example/ask -d '{"department":"billing","question":"What is an integration platform?"}'
```

Expected output - the cost of this call is in the response:

```json
{"answer": "An integration platform connects...", "cost_usd": 0.000141}
```

And the server log now has the line that accounting tools aggregate:

```text
llm-cost department=billing input_tokens=14 output_tokens=213 cost_usd=0.000141
```

Summing a day's spend per department is one command over the log:

```bash
grep -h 'llm-cost' ~/env/qs-1/server1/logs/server.log | \
    awk -F'department=| input' '{print $2}' | sort | uniq -c
```

The REST channel:

```yaml
channel_rest:
  - name: example.ai.ask-with-accounting
    service: example.ai.ask-with-accounting
    url_path: /example/ask
```

## Chat costs {#chat-costs}

In a [chat](https://zato.io/docs/ai/llm/chat.html), `input_tokens` covers everything the provider received - the history included. The number grows turn by turn until trimming caps it, which is the cost ceiling `max_history_turns` sets.

## Failure behavior {#failure-behavior}

A failed call raises before any tokens are consumed on the reply, so nothing is logged for it here - but the failure itself is in the [audit log](https://zato.io/docs/ai/llm/connections.html#the-audit-trail) with its duration and error. The one case that spends tokens without a log line is a call that succeeds while the service crashes later - which is why the cost line above is written immediately after the call, before anything else can fail.

## See also {#see-also}

- [Invoking LLMs](https://zato.io/docs/ai/llm/invoke.html) - The usage dictionary on every response
- [Cost and limits](https://zato.io/docs/ai/cost-and-limits.html) - The limits that cap what a call may spend
- [Provider failures](https://zato.io/docs/ai/examples/provider-failures.html) - What failed calls cost and where they are recorded