Downtime and recovery

Interfaces go down - maintenance windows, network incidents, a downstream system offline for an upgrade. What matters is what happens to the messages in the meantime and how the recovery avoids the two failure modes: losing a message and processing one twice.

While you are down

An MLLP sender does not consider a message delivered until the acknowledgment arrives. While your side is unreachable, the sending system keeps the message and retransmits - queueing on the sender's side is how the protocol itself bridges an outage.

Two things follow from that:

• The backlog arrives on its own. When your side is back, senders deliver what they queued - there is no manual step to request it.
• Retransmissions overlap with what already arrived. A sender that never received an ACK for a message you did in fact process will send that message again during the catch-up.

Retransmissions of already-processed messages are what deduplication exists for. Each message's MSH-10 control ID is remembered for the configured window - a retransmission of an already-processed message is answered with an immediate AA and your service is not invoked again. The window has to outlast the longest outage plus the sender's retransmission policy - the default is 14 days.

After you are back

• Watch the catch-up land. The audit log filtered to the channel shows the burst - the activity strip makes the backlog visible as a spike, and each message records the acknowledgment it received.
• Confirm nothing is left waiting. The Outstanding filter lists the exchanges whose acknowledgment has not arrived, oldest first - after a recovery it should drain to empty.
• Reconcile the outage window. The sender's count of messages emitted during and after the outage against the audit log's count of messages received - the same reconciliation as at go-live, applied to one day.

When a destination is down

The direction reverses when a system you send to is offline: your side holds the recorded messages and the destination misses its copies.

Every sent message is stored in the audit log with its full body, not only the metadata - so the recovery is resubmission, from the same screen where the failures are listed:

• Resending sends the stored payload through the same outgoing connection again - the right action when the failure was on the far side or in between, which is what an outage is.
• A hop resend targets one destination of several. One incoming message delivered to three systems, one of which was down, is recovered by sending that one destination its copy - nobody who already has the message gets it twice.
• The classification column helps with the order of work. Failures whose errors look like timeouts or refused connections are marked transient - resubmitting them as-is can work. Failures pointing at the message content are marked permanent and need a fix before another attempt.

A message that has already been resubmitted says so instead of offering the action again, which is what stops the same message going out twice during a hurried recovery.

Knowing it happened at all

A feed that goes silent produces no errors on your side, so silence needs its own detection. Alert rules over the audit log cover both directions - failures that spike and traffic that stops - and each notification links straight to the events behind it, with the resubmit confirmation one click away.

Learn more