# Cost and limits

Cap AI cost and traffic with token limits, timeouts and size caps.

AI traffic costs money per token and fails when it exhausts capacity - context windows, budgets and thread pools alike. The limits below cap both, each with its default and a link to the page that describes it. All of them are configuration - no service code changes when a limit does.

## LLM limits {#llm-limits}

Set per [LLM connection](https://zato.io/docs/ai/llm/connections.html), applying to every call through it:

| Limit | Default | What it does |
| --- | --- | --- |
| `max_tokens` | 1024 | The most tokens the model may generate per reply - the direct per-call cost ceiling for output |
| `timeout` | 60 s | How long one call may wait for the provider - a slow provider costs thread time even when it costs no tokens |
| `pool_size` | 50 | How many HTTP clients the pool keeps ready - the ceiling on concurrent calls through this connection |
| `max_history_turns` | 20 | How many past turns of a [chat](https://zato.io/docs/ai/llm/chat.html) are sent with each call - the input-token ceiling of a conversation, since older turns never leave the platform |
| `chat_expiry` | 86,400 s | How long a chat's history is kept after its last message - a storage limit with no effect on tokens, which determines how long a conversation can be resumed |

The per-call spend is visible in every response through the `usage` dictionary, described under [invoke](https://zato.io/docs/ai/llm/invoke.html).

## MCP limits {#mcp-limits}

Set per [MCP gateway](https://zato.io/docs/ai/mcp/index.html):

| Limit | Default | What it does |
| --- | --- | --- |
| `invoke_timeout` | 90 s | How long one `tools/call` may run - a hung service otherwise keeps the agent waiting and the worker busy |
| `session_ttl` | 1,800 s | How long a [session](https://zato.io/docs/ai/mcp/index.html#endpoint-behavior) may stay idle - expired sessions free their per-identity slots |
| `max_response_size` | off | The token-denominated cap on tool responses - the context-window protection, with truncate and block modes described under [response controls](https://zato.io/docs/ai/mcp/response-controls.html) |
| `min_size_threshold` | off | Responses estimated below this many tokens skip all shaping - small responses pay no processing cost |
| `characters_per_token` | 4.0 | The estimation ratio the two caps above are computed with, described under [response controls](https://zato.io/docs/ai/mcp/response-controls.html#token-estimation) |

[Rate limits](https://zato.io/docs/ai/mcp/rate-limits.html) - caps on how often each agent may call - are configured on the security definition, not the gateway, and are covered on their own page.

## Configuration {#configuration}

All LLM limits are on the connection's Dashboard form and in the `llm` section of [enmasse](https://zato.io/docs/ai/enmasse.html). The MCP size caps are on the wizard's step 02, while `session_ttl` and `invoke_timeout` are set through the `mcp_gateway` enmasse section.

## See also {#see-also}

- [Rate limits](https://zato.io/docs/ai/mcp/rate-limits.html) - Caps on how often each agent may call, per identity
- [Response controls](https://zato.io/docs/ai/mcp/response-controls.html) - The size caps and token estimation in full
- [LLM connections](https://zato.io/docs/ai/llm/connections.html) - The form and fields the LLM limits are set on
- [GitOps](https://zato.io/docs/ai/enmasse.html) - The same limits declared in YAML

## Learn more {#learn-more}

- [MCP tutorial](https://zato.io/tutorials/mcp/01.html) - Expose Python services as AI tools and connect Claude Code in minutes
- [AI Integrations overview](https://zato.io/docs/ai/) - Both directions of AI traffic in one platform, and where to start
- [MCP gateway reference](https://zato.io/docs/ai/mcp/) - Configuration, endpoint behavior and the governance controls
- [Tool schemas](https://zato.io/docs/ai/mcp/tool-schemas.html) - How a service's docstring and declared I/O become its tool definition
- [MCP gateway security](https://zato.io/docs/ai/mcp/security.html) - API keys, Basic Auth and bearer tokens for AI agents
- [MCP response controls](https://zato.io/docs/ai/mcp/response-controls.html) - PII removal, prompt-injection safeguards and token-denominated size caps
- [MCP audit log](https://zato.io/docs/ai/mcp/audit-log.html) - What each agent did, when and with what outcome - payloads never recorded
- [MCP alerts](https://zato.io/docs/ai/mcp/alerts.html) - Failing backends, agents stuck in a loop and gateways with too many tools
- [GitOps for AI integrations](https://zato.io/docs/ai/enmasse.html) - LLM connections and MCP gateways as YAML in version control
- [AI examples](https://zato.io/docs/ai/examples/) - Complete, runnable scenarios - one real-world need per file