Alerting in enmasse

Alert thresholds and notification targets kept in YAML under the alert_rules and alert_notifications keys.

Alert thresholds and notification targets are part of enmasse, so they live in your project's YAML alongside everything else and reach every environment the same way.

Two top-level keys hold them: alert_rules for the thresholds and alert_notifications for where alerts are delivered.

Note: Enmasse changes the rules Zato ships with. It cannot add new rules, and there is nothing to delete - a rule you do not want is switched off rather than removed. An entry in alert_rules is therefore always an update to one of the 11 shipped rule types.

A worked example

alert_rules:
  - type: rest
    is_active: true
    consecutive_failures: 3
    error_rate: 20
    alert_threshold: 40
    max_latency: 7000
    use_llm: true

  - type: sql
    is_active: true
    error_rate: 5
    max_query_time: 2000

  - type: file_transfer
    is_active: true
    warning_failures: 12
    critical_failures: 24
    test_transfers: false

  - type: common
    is_active: true
    certificate_warning: 30

alert_notifications:
  webhook_url: https://example.atlassian.net/automation/webhooks/abc
  email_to: ops@example.com
  email_from: alerts@example.com
  dashboard_url: https://zato.example.com

Every field is optional except type - an entry sets what it names and leaves the rest at whatever the environment already has.

alert_rules

Each entry has two meta fields and then the values of one rule type.

NameRequiredDescription
typeYesWhich rule type this entry configures, one of the 11 values below
is_active---Whether every rule of this type is on. This is the card's header checkbox

The type values are rest, sql, llm, mcp, microsoft, email, odoo, file_transfer, scheduler, channels and common. The rest type covers outgoing SOAP as well as REST.

The fields of each type

TypeFields
restconsecutive_failures, error_rate, alert_threshold, max_latency, use_llm
sqlconsecutive_failures, error_rate, alert_threshold, max_query_time, use_llm
llmconsecutive_failures, error_rate, alert_threshold, warning_latency, critical_latency, use_llm
mcpconsecutive_failures, error_rate, alert_threshold, max_tool_call_time, use_llm
microsoftconsecutive_failures, error_rate, alert_threshold, health_alerts, max_call_time, use_llm
emailconsecutive_failures, error_rate, alert_threshold, auth_failures, use_llm
odooconsecutive_failures, error_rate, alert_threshold, auth_failures, max_call_time, use_llm
file_transferconsecutive_failures, warning_failures, alert_threshold, critical_failures, test_transfers, use_llm
schedulererror_rate, alert_threshold, overdue_multiplier, start_delay, use_llm
channelserror_rate
commoncertificate_warning, outstanding_backlog, feed_silence

What each one means, and its shipped default, is on the rules page. Three of them are true or false rather than numbers: use_llm, health_alerts and test_transfers.

Percentages are typed as percentages here too.error_rate: 10 means ten percent, exactly as on the screen. error_rate: 0.1 means one tenth of one percent and will alert on nearly everything.

alert_notifications

A single mapping, not a list - these are the environment's delivery settings.

NameDescription
webhook_urlURL that the webhook action POSTs alerts to
email_toAddress alerts are sent to
email_fromAddress alerts are sent from
dashboard_urlBase URL of your Dashboard, used to make the link in every alert absolute

These are addressing values only. Email, Slack and Teams themselves deliver through the connections named default.alerts.notifications, which ship inactive, and notifications covers what has to be activated.

Exporting what you have

An export writes both keys out with the values currently in force, so exporting an environment nobody has changed yet shows you the shipped defaults as YAML.

Learn more