# Quiet monitor contract

Version: 1.0
Updated: 2026-10-02
Method: https://suhailmohebi.com/notes/the-quiet-monitor/

Purpose: build a monitor that stays quiet without going blind, and interrupts a person only when a decision is due.

Output: a one-page contract covering one monitor’s decision, record, states, routing, liveness, heartbeat and retirement.

## Use when

You are building or reviewing a scheduled check or feed watcher, or one has been green so long nobody knows what turns it red.

## Quick start

1. Name the decision the monitor serves and the threshold that justifies interrupting someone.
2. Give every source an honest state, and assert the subject exists before asserting it is healthy.
3. Wrap the job in a heartbeat so silence from the monitor itself raises an alert.

Unknown is not unchanged. A broken collector must never produce an all-clear.

## 1. Decision and threshold

> This monitor exists so that [person or role] can decide [decision] within [time]. It interrupts only when [change] is seen
> with [confidence] and matters for [consequence].

- Interrupt now when:
- Batch into a digest when:
- Record silently when:
- Duplicate rule (same conclusion already sent):

Signal = change × confidence × consequence − duplication.

## 2. Observation record

Append one record per source per run, never overwrite, and derive changes from the sequence. Fields: `observed_at`, `first_seen_at`, `source`, `retrieval_status`, `state` and `value` (the normalised value or hash compared between runs).

Collect, normalise, detect, interpret and deliver each keep a health state. A vanished item is a candidate change until repetition, source health or a second source confirms it.

## 3. Source states

| State | Meaning | Triggers |
|---|---|---|
| OBSERVED | a current response was captured and validated | change detection |
| UNCHANGED | healthy, no material delta | record silently |
| CHANGED | a normalised delta passed the change rule | route by section 4 |
| DEGRADED | responded, but stale or incomplete | run incomplete: retry, then no all-clear, no ping |
| UNKNOWN | no trustworthy current observation: failed, blocked, unparseable | run incomplete: retry on the same bounded path, then no all-clear, no ping |
| DECOMMISSIONED | the subject no longer exists | a person decides: retire, repoint or keep |

OBSERVED, DEGRADED, UNKNOWN and DECOMMISSIONED are source health (`retrieval_status`); UNCHANGED and CHANGED are the change result (`state`) and exist only for an OBSERVED source. A valid empty result counts as OBSERVED only after the liveness assertion in section 5 passes.

## 4. Severity separate from confidence

| Evidence | Consequence | Route |
|---|---|---|
| Weak | High | collect faster, seek corroboration, label the uncertainty |
| Strong | High | interrupt with source, timestamp, delta and the decision required |
| Strong | Low | batch into the next digest |
| No material delta | Any | record silently; keep source-health evidence |

Models may classify, cluster or summarise changes, but never create observations or send alerts. Facts, scoring inputs, deduplication keys, budgets and delivery state stay deterministic, and a model never upgrades “possibly related” to “caused by”.

## 5. Subject-liveness assertion

Before any health check, assert the subject: the page resolves, the endpoint exists, the stream carried at least
[minimum volume] in the window.

| Subject | Failures seen | Honest state |
|---|---|---|
| Exists, active | None | Liveness passed: the measurement means something |
| Exists, no volume | None | UNKNOWN: nothing was exercised |
| Gone | None | DECOMMISSIONED: a person decides to retire, repoint or keep the monitor |

Input that would make this check fail today: [input]. If no input can, the check is decoration.

## 6. Heartbeat for the monitor itself

A heartbeat service alerts when no ping arrives within [interval] plus [grace]; base the grace on measured run times. The monitor exits non-zero on failed liveness, an UNKNOWN or DEGRADED required source, or an interrupt whose delivery ended failed or dead-lettered, so incomplete runs send no ping.

```bash
#!/usr/bin/env bash
# heartbeat-wrap.sh: run the monitor; ping the heartbeat only if it succeeded.
# Usage: HEARTBEAT_URL="https://heartbeat.example.com/ping/ID" ./heartbeat-wrap.sh ./monitor.sh
set -u
: "${HEARTBEAT_URL:?is not set; refusing to run unwatched}"
if [ "$#" -eq 0 ]; then
  echo "usage: $0 command [args...]" >&2
  exit 64
fi

"$@"
status=$?
if [ "$status" -ne 0 ]; then
  echo "monitor exited $status; no heartbeat sent" >&2
  exit "$status"
fi
# -f: HTTP errors fail; -m 10: per-attempt cap; --retry 3: transient errors only.
if ! curl -fsS -m 10 --retry 3 "$HEARTBEAT_URL" >/dev/null; then
  echo "monitor succeeded but the heartbeat ping failed" >&2
  exit 75
fi
```

Keep the URL out of the repository and out of shell history: set it in the scheduler’s environment or a file only the job can read. Anyone holding it can fake success. Test the alarm by skipping one run.

If your heartbeat service accepts an explicit failure signal (Healthchecks.io: append `/fail` or `/<exit status>` to the ping URL), send it on a non-zero exit so the alarm fires at once instead of after the grace period.

## 7. Fail-closed rules

- [ ] A failed, blocked or stale source makes the run incomplete; zero findings from a broken feed is an observer failure.
- [ ] Missing required evidence keeps an item on watch; it never resolves it.
- [ ] A failed run is kept as evidence and retried on the same bounded path, never reconciled as clean.
- [ ] An interrupt that could not be delivered fails the run; a failed or dead-lettered alert is never a clean run.
- [ ] The monitor only observes. Active testing or remediation is a separate, authorised task.

## 8. Decision receipt

```text
WHAT CHANGED: [exact normalised delta]
EVIDENCE: [source] | observed_at [time] | first_seen_at [time] | corroboration [repeat or second source]
STATE: confidence [weak/strong] | severity [low/high] | source health [state] | duplicate [no / of receipt N]
DECISION: [why this crossed the threshold; what must be decided, by whom, by when]
LIMIT: [what remains unknown or unavailable]
DELIVERY: queued / sent / acknowledged / failed / dead-lettered
```

A sent alert proves a request travelled, not that the right person saw it; track it to acknowledged. An interrupt not acknowledged within [time] is re-sent to the next person in the rota.

## 9. Retirement and maintenance tail

Remove a subject and its watchers in the same change. After any removal, audit every recurring job:

| Watcher | Subject | Exists with volume? | Input that fails it today | Disposition |
|---|---|---|---|---|
|  |  | Yes / No |  | Retire / Repoint / Keep: [reason] |

Retire the heartbeat check with its monitor, or its absence alert fires for a job you meant to stop.

## Worked example

A nightly 02:00 UTC monitor checks a public advisory feed against a service’s lockfile so on-call can decide within two hours whether to block the 06:00 automatic deploy. It interrupts only for an exact package match inside the affected range.

| Source | Retrieval | State | Route |
|---|---|---|---|
| Advisory feed | HTTP 200, 1,204 records | OBSERVED, CHANGED: new advisory for `libfoo` | strong and high: interrupt |
| Lockfile at `abc1234` | read, 212 packages | OBSERVED, UNCHANGED | record silently |
| Liveness | records returned; lockfile above 50 packages | passed | evaluate the change rule |

```text
WHAT CHANGED: new advisory ADV-0001 for npm package libfoo: affected <2.3.4, fixed in 2.3.4
EVIDENCE: advisory feed | observed_at 2026-10-02 02:00 UTC | first_seen_at same run | corroboration: lockfile pins libfoo 2.3.1
STATE: confidence strong (exact package and version) | severity high | sources OBSERVED | duplicate no
DECISION: installed version is inside the affected range; on-call decides by 04:00 UTC whether to block the 06:00 deploy
LIMIT: reachability of the affected code not assessed; nothing tested actively
DELIVERY: sent 02:01 UTC; acknowledged 02:14 UTC
```

Next night the feed times out after retries: UNKNOWN, run incomplete, exit 2, no ping. With a 24-hour interval and 1-hour grace, the heartbeat alert fires near 03:00 instead of a silent “no new advisories”.

## Stop if

- No decision or decision owner can be named.
- Someone asks for an all-clear while a required source is UNKNOWN or DEGRADED.
- Liveness fails or the subject is gone.
- A model output would become an observation or send an alert.
- `HEARTBEAT_URL` is unset or pings stop arriving: fix the watch before trusting any result.

## Sources

- Healthchecks.io: Monitor Shell Scripts with Healthchecks.io — https://healthchecks.io/docs/bash/ (checked 2026-10-02)
- Healthchecks.io: Healthchecks.io Documentation — https://healthchecks.io/docs/ (checked 2026-10-02)
- curl: curl man page — https://curl.se/docs/manpage.html (checked 2026-10-02)