# Health checks

The ping endpoints of each component, and proactive checks of outgoing connections.

An outgoing REST, SOAP or FHIR connection can ping itself on a schedule. The result of each ping is recorded in the audit log.

A broken connection can only be noticed when somebody is calling it. On a connection used once a month, a broken one and a healthy one look identical until the day you need it. A health check gives it a heartbeat to fail.

Outgoing REST, outgoing SOAP and outgoing FHIR connections have health checks. No other connection type has them, and channels have none at all - a channel is called by somebody else, so there is nothing for Zato to ping.

## The ping endpoints {#the-ping-endpoints}

Two similar-sounding features point in opposite directions:

- A **ping endpoint**, in this section, is called by your monitoring or your orchestrator and reports that Zato itself is up.
- A **health check**, the rest of this page, is run by Zato, on a schedule, against your outgoing connections, and reports that the systems Zato calls are up.

Each component offers an HTTP ping endpoint - a dedicated address that external monitoring tools can use to discern when the component is operational:

| Component | TCP port | URL Path | Method | Expected result |
| --- | --- | --- | --- | --- |
| Server | 11223 | `/zato/ping` | GET | 200 OK with a JSON confirmation |
| Dashboard | 8183 | `/` | GET | 302 redirect to `/zato` |

To ping a server:

```text
$ curl -XGET http://localhost:11223/zato/ping
{"is_ok":true, "cid":"20260921-180542-8418-c1e5d65e18f62730a"}
$
```

To ping an instance of Dashboard - it answers with a redirect to its login page:

```text
$ curl -I http://localhost:8183/
HTTP/1.1 302 Found
Location: /zato
```

All the other ports an environment listens on are in the [default ports reference](https://zato.io/docs/admin/ports.html).

### Using the ping endpoint as a probe {#using-the-ping-endpoint-as-a-probe}

The server's ping endpoint is what container orchestrators should point their probes at.

In Kubernetes:

```yaml
readinessProbe:
  httpGet:
    path: /zato/ping
    port: 11223
  initialDelaySeconds: 60
  periodSeconds: 10
  failureThreshold: 3

livenessProbe:
  httpGet:
    path: /zato/ping
    port: 11223
  initialDelaySeconds: 120
  periodSeconds: 30
  failureThreshold: 5
```

In Docker Compose:

```yaml
healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost:11223/zato/ping"]
  interval: 30s
  timeout: 5s
  retries: 3
  start_period: 120s
```

> **Note:** Give the start-up enough time in `start_period` and `initialDelaySeconds`. A liveness probe that runs out of patience before the container is ready restarts it forever.

## How it is recorded {#how-it-is-recorded}

A ping goes into the [audit log](https://zato.io/docs/admin/audit-log/index.html) like any other call, under its own source, `REST checks`, `SOAP checks` or `FHIR checks`, and is counted separately from the connection's own traffic.

Check events are kept for 7 days, rather than the 30 days the connection's traffic gets. The [audit log retention settings](https://zato.io/docs/admin/audit-log/index.html) can change this per source.

> **A check is recorded whether or not the connection's own audit log is on.** The `Audit log` checkbox on the connection's Config tab controls whether the connection's traffic is recorded. Its health check writes its pings regardless.

## Turning one on {#turning-one-on}

On an outgoing REST connection:

1. In Dashboard, go to `Connections` -> `Outgoing` -> `REST` and edit the connection.
2. Open the `Alerts` tab.
3. Under `Health check`, click the `Schedule` line and set `Ping every` to a number and its unit - seconds, minutes, hours or days.
4. Save. Checks begin immediately, there is no start date to set.

On an outgoing SOAP connection:

1. In Dashboard, go to `Connections` -> `Outgoing` -> `SOAP` and edit the connection.
2. Open the `Health check` tab.
3. Set `Run every` to a number and pick its unit.
4. Save.

On an outgoing FHIR connection:

1. In Dashboard, go to `Connections` -> `Outgoing` -> `HL7` -> `FHIR` and edit the connection.
2. Open the `Alerts` tab.
3. Under `Health check`, click the `Schedule` line and set `Ping every` to a number and its unit.
4. Save.

To remove a health check, clear the number.

> **Note:** The unit starts at `minutes`. Typing `5` and saving without changing the unit gives you a ping every five minutes.

## What it sends {#what-it-sends}

The connection's own ping, to the connection's own address. There is no separate health check URL to configure, and nothing to build on the remote side.

A FHIR connection's ping is a `GET` of the server's `CapabilityStatement` - the resource every FHIR server serves at `[base]/CapabilityStatement` - so the points below about the method and the URL path apply to REST and SOAP connections alone.

- The method is `HEAD` by default. It is the connection's `Ping method` on the Config tab, so change it to `GET` if the remote system answers 405 to a HEAD.
- Anything from 400 up is a failure, and so is any exception - a timeout, a refused connection, a TLS problem.
- The ping uses the connection's own timeout and its own retry policy, so a connection configured to retry will retry the ping too before calling it a failure.

> **Note:** Path parameters are not filled in for a ping. A connection whose URL path is `/api/items/{item_id}` is pinged at that literal address, which will usually answer 404 and read as a permanently broken connection.

## What a failed check does {#what-a-failed-check-does}

Nothing beyond recording the result. The connection is not disabled and the next call through it proceeds as before. Two things stop checks running at all:

- Disabling a connection also stops its health checks, so an inactive connection is not being watched.
- Checks run as scheduler jobs, so a container started with `Zato_Start_Scheduler=False` runs none, which is indistinguishable from everything being fine.

## In enmasse {#in-enmasse}

The same two keys live on the connection in [enmasse](https://zato.io/docs/admin/enmasse.html):

```yaml
outgoing_rest:

  - name: inventory.items
    host: https://inventory.example.com
    url_path: /api/items
    ping_method: HEAD

    health_check_run_every: 5
    health_check_run_unit: minutes
```

`health_check_run_every` is the only one enmasse insists on. Left out, `health_check_run_unit` defaults to `minutes`. An `outgoing_soap` or an `outgoing_fhir` connection takes the same two keys.

Valid values for the unit are `seconds`, `minutes`, `hours` or `days`.

## Learn more {#learn-more}

- [Production checklist](https://zato.io/docs/admin/production-checklist.html) - What to confirm before an environment takes real traffic
- [Audit log](https://zato.io/docs/admin/audit-log/) - Every request, response and error, and what you can do with them afterwards
- [Logging](https://zato.io/docs/admin/logging.html) - What is written where, and how to ship it to Splunk, Loki, Elastic or Datadog
- [GitOps with enmasse](https://zato.io/docs/admin/enmasse.html) - Keep the whole environment in git as YAML and import it anywhere