# Chat with memory

Build a support chat that keeps each user's conversation separate, with history the platform manages.

A support chat where each user has their own conversation - the model receives this user's earlier turns and never another user's, history survives restarts, and a skill supplies the assistant's standing instructions without entering any conversation's history.

**Prerequisites.** An [LLM connection](https://zato.io/docs/ai/llm/connections.html) named `Support LLM` and a [skill](https://zato.io/docs/ai/skills/index.html) named `support-agent` - both are defined under [the configuration](https://zato.io/docs/ai/examples/chat-with-memory.html#the-configuration) and [the skill](https://zato.io/docs/ai/examples/chat-with-memory.html#the-skill).

## The service {#the-service}

```python
# -*- coding: utf-8 -*-

# Zato
from zato.server.service import Service

class SupportChat(Service):
    """ A per-user support chat - each user_id is its own conversation.
    """
    name = 'example.ai.support-chat'

    input = 'user_id', 'text'
    output = 'reply'

    def handle(self):

        conn = self.llm['Support LLM']

        # The user's own id is the chat id - this is the whole isolation mechanism.
        # Two users can never share a conversation because they never share an id.
        chat_id = 'support.' + self.request.input.user_id

        response = conn.chat(
            self.request.input.text,
            chat_id=chat_id,
            skill='support-agent',
        )

        self.response.payload.reply = response['text']
```

## Run the service {#run-the-service}

Call the service twice for the same user:

```bash
curl localhost:11223/example/support-chat -d '{"user_id":"alice","text":"My order 4711 has not arrived"}'
curl localhost:11223/example/support-chat -d '{"user_id":"alice","text":"What was my order number again?"}'
```

The second reply names order 4711 - the model received the first turn along with the second question. Now call it as another user:

```bash
curl localhost:11223/example/support-chat -d '{"user_id":"bob","text":"What was my order number again?"}'
```

The model has nothing to answer from - Bob's `chat_id` is `support.bob`, a conversation with no turns in it yet. That is the expected output: memory per user, not per service.

## History trimming {#history-trimming}

The connection below sets `max_history_turns` to 5, so each call sends at most the last 5 turns - a turn is one user message plus the assistant's reply. Ask ten questions numbered one to ten, then ask:

```bash
curl localhost:11223/example/support-chat -d '{"user_id":"alice","text":"What was my first question?"}'
```

The model cannot answer - questions one through five have been trimmed from what goes to the provider. They are still stored, and raising `max_history_turns` on the connection makes the next call send more of them - the semantics are under [history trimming](https://zato.io/docs/ai/llm/chat.html#history-trimming).

## Expiry {#expiry}

With `chat_expiry` of 3600 below, a conversation quiet for an hour starts afresh - the expiry semantics are under [history storage](https://zato.io/docs/ai/llm/chat.html#history-storage).

## The skill {#the-skill}

The skill accompanies every call, as [use a skill](https://zato.io/docs/ai/skills/index.html#use-a-skill) describes - `config/repo/skills/support-agent/SKILL.md`:

```markdown
---
name: support-agent
description: How to answer customer support questions
---

You are a support agent for an electronics retailer.

* Answer in the customer's language.
* Keep replies under three sentences.
* When you do not know, say so and point to support@example.com.
```

## The configuration {#the-configuration}

The [enmasse](https://zato.io/docs/ai/enmasse.html) YAML - the connection, with its key in an environment variable, and the REST channel the `curl` calls above go through:

```yaml
llm:
  - name: Support LLM
    model: gpt-4o-mini
    address: https://api.openai.com/v1
    secret: Zato_Enmasse_Env.Support_LLM_Key
    max_history_turns: 5
    chat_expiry: 3600

channel_rest:
  - name: example.ai.support-chat
    service: example.ai.support-chat
    url_path: /example/support-chat
```

## Failure behavior {#failure-behavior}

A provider outage mid-conversation raises an exception from `chat` and the user's message is not written into the history - the conversation is never left with a question the model never saw. The failed call is in the [audit log](https://zato.io/docs/ai/llm/connections.html#the-audit-trail) and retrying the same request continues the conversation exactly where it was. Two simultaneous messages from the same user never race either - calls to one `chat_id` are serialized, as described under [concurrent calls](https://zato.io/docs/ai/llm/chat.html#concurrent-calls).

## See also {#see-also}

- [Multi-turn conversations](https://zato.io/docs/ai/llm/chat.html) - The chat call, history storage, trimming and concurrency
- [Skills](https://zato.io/docs/ai/skills/index.html) - The instruction files that supply the system context
- [Token usage](https://zato.io/docs/ai/examples/token-usage.html) - What each call of a conversation costs
- [Provider failures](https://zato.io/docs/ai/examples/provider-failures.html) - A service that survives its provider going down