Alert rules

Four shapes repeat across the connection types, and a handful of rules cover subjects of their own.

The four shapes

Connection down

A run of consecutive failures with no success in between. The default is 3 and the severity is critical, so it is delivered every time it is measured rather than being suppressed as a repetition.

This is the rule that catches an outage, and it is also the rule that stays silent on a connection nobody is calling. A connection with no traffic produces no failures, so nothing is measured. Turning on a health check gives it a heartbeat to fail.

Error rate

The share of recent traffic that ended in an error, over a five-minute window. The default is 10 percent and the severity is a warning.

A minimum of 10 events applies before the rule can trip, so a connection called twice, once unsuccessfully, does not report a 50 percent error rate.

Latency

An average response time above a ceiling. The ceiling differs per connection type. The severity is a warning.

Error rate with diagnosis

The same error-rate measure at a higher threshold, 25 percent by default. Instead of sending a plain notification, it invokes the LLM diagnosis action, which looks at the failing events and writes up what it thinks is wrong. The severity is critical.

This is the rule the LLM switch on each card turns on and off.

Every threshold

One row per type, the shipped defaults. A dash means the type has no rule of that shape.

Every error rate above requires at least 10 events in the window before it can trip.

The REST row covers outgoing REST and outgoing SOAP connections, and it covers their health checks as well - a health check is measured by the same rules on the same thresholds as the traffic the connection delivers, just counted separately from it.

Single-subject rules

The test transfer rule writes a test file to a remote system, so it ships off. The Test transfers switch on the File transfer card turns on both the rule and the job that writes the file.

The config screen

Monitoring -> Alert rules in Dashboard shows one card per rule type - REST and SOAP, SQL, LLM, MCP, Microsoft cloud, Email, Odoo, File transfer, Scheduler, Channels, and Common, which holds the certificate, backlog and feed rules that belong to no one connection type.

Each card shows the values its type's rules are driven by. Most cards carry Consecutive failures, Error rate, Alert threshold and a latency ceiling, and the cells differ where the type does - LLM has separate warning and critical latencies, Email and Odoo add Auth failures, File transfer has warning and critical transfer failure counts, and Scheduler has its overdue multiplier and start delay.

Three controls sit beside the values:

• The checkbox in a card's header turns every rule of that type on or off at once.
• The LLM cell is the switch of the type's diagnosis rule.
• The Test transfers cell on the File transfer card turns on both the test transfer rule and the scheduler job that runs the test transfers - the two always move together.

The row below the cards holds where the alerts go - Slack webhook, Teams webhook, Webhook URL, Email connection, Email to, Email from and Dashboard URL. What each value means and how a delivery uses it is on notifications.

Changing a threshold

1. In Dashboard, go to Monitoring -> Alert rules.
2. Find the card for the type you want, for instance REST and SOAP.
3. Click the value you want to change. Consecutive failures, Error rate, Alert threshold and the latency cell are the four you will normally touch.
4. Type the new value and save. The change is stored at once and the next sweep already measures with it - there is no restart. A save that changes nothing stores nothing.

In enmasse

The same values live under the alert_rules key of your enmasse file, which is how you keep them in version control and apply them to every environment. Enmasse changes the shipped rules and cannot add new ones.

Learn more