Cost and limits
Cap AI cost and traffic with token limits, timeouts and size caps.
AI traffic costs money per token and fails by flooding - context windows, budgets and thread pools alike. The limits below cap both, each with its default and a link to the page that owns it. All of them are configuration - no service code changes when a limit does.
LLM limits
Set per LLM connection, applying to every call through it:
| Limit | Default | What it does |
|---|---|---|
max_tokens | 1024 | The most tokens the model may generate per reply - the direct per-call cost ceiling for output |
timeout | 60 s | How long one call may wait for the provider - a slow provider costs thread time even when it costs no tokens |
pool_size | 50 | How many HTTP clients the pool keeps ready - the ceiling on concurrent calls through this connection |
max_history_turns | 20 | How many past turns of a chat are sent with each call - the input-token ceiling of a conversation, since older turns never leave the platform |
chat_expiry | 86,400 s | How long a chat's history is kept after its last message - storage, not tokens, but the knob that decides how long a conversation can be resumed |
The per-call spend is visible in every response through the usage dictionary, described under invoke.
MCP limits
Set per MCP gateway:
| Limit | Default | What it does |
|---|---|---|
invoke_timeout | 90 s | How long one tools/call may run - a hung service otherwise keeps the agent waiting and the worker busy |
session_ttl | 1,800 s | How long a session may stay idle - expired sessions free their per-identity slots |
max_response_size | off | The token-denominated cap on tool responses - the context-window protection, with truncate and block modes described under response controls |
min_size_threshold | off | Responses estimated below this many tokens skip all shaping - small responses pay no processing cost |
characters_per_token | 4.0 | The estimation ratio the two caps above are computed with, described under response controls |
Rate limits - caps on how often each agent may call - are configured on the security definition, not the gateway, and are covered on their own page.
Configuration
All LLM limits are on the connection's Dashboard form and in the llm section of enmasse. The MCP size caps are on the wizard's step 02, while session_ttl and invoke_timeout are set through the mcp_gateway enmasse section.
See also
| Feature | What it does |
|---|---|
| Rate limits | Caps on how often each agent may call, per identity |
| Response controls | The size caps and token estimation in full |
| LLM connections | The form and fields the LLM limits are set on |
| GitOps | The same limits declared in YAML |