How I built a fail-closed vulnerability watcher

§1The honest output can be “incomplete”

A fail-closed vulnerability watcher does not treat silence as safety. If an authoritative feed fails, if a package name only looks similar, or if the installed version is unknown, the run stays incomplete or the item stays on watch. It does not quietly become “not affected.”

That sounds conservative because it is. The purpose of the system is not to maximize the number of alerts or produce a comforting green dashboard. It is to turn public vulnerability intelligence into a small set of decisions whose evidence can be inspected.

§2A CVE list is not a monitor

A vulnerability list answers “what has been reported?” An operational monitor has to answer harder questions: Which records changed? Which ones map to software I actually run? Is the match exact? Is my version inside the affected range? Is exploitation known, predicted, or merely possible? What evidence is still missing?

The public sources each contribute a different kind of fact. The CISA Known Exploited Vulnerabilities catalog is an input for prioritising vulnerabilities with evidence of exploitation. The NVD vulnerability API supplies CVE records, metrics, references, and applicability data. FIRST EPSS estimates the probability of exploitation. The GitHub Advisory Database adds ecosystem and package context.

None of those sources can, by itself, prove that one of my systems is exposed. The local inventory and version evidence close that loop.

§3The evidence pipeline

I designed the watcher as a sequence of evidence gates. Every stage narrows the question, and every transition leaves a record.

Normalization is deliberately loss-aware. I keep the source, retrieval time, advisory identifiers, affected ranges, and enough raw evidence to explain how a decision was reached later. A neat table is useful; a reproducible decision is better.

§4Feed health is part of the result

The easiest dangerous mistake is converting a network or parser failure into zero findings. If a required source returns nothing because DNS failed, an API changed, or the runtime was denied network access, zero is not a security conclusion. It is an observer failure.

Those states must remain distinct in the artifact and the operator summary. If a feed fails, I preserve the failed run, mark the catalog incomplete, and retry the same bounded collection path when the environment is available. I do not reconcile the failed artifact as a clean scan.

§5Product identity comes before urgency

Names create false confidence. A recent match connected CVE-2021-21315 to a Node.js inventory entry because the advisory described a “System Information Library for Node.JS.” The affected component is the systeminformation npm package, not the Node.js runtime. The words overlap; the products do not.

The same failure appears when an advisory for an integration package is mistaken for the application it connects to. “n8n-MCP” and “n8n” are not interchangeable inventory identities. Severity cannot rescue a bad join.

This is where an agent helps most as a skeptical reconciler: it can compare aliases, source descriptions, package ecosystems, and previous decisions. The deterministic gate still decides whether the exact identity evidence exists.

§6The version-evidence gate

Once the product matches, the next question is not “is the CVE severe?” It is “what version is actually installed, and does an authoritative advisory place that version inside the affected range?”

product + affected range + installed version + safe objective → review candidate

A missing installed version is not a minor metadata gap. It blocks the key inference. The watcher keeps the item visible and tells me what evidence to collect next.

§7Public systems stay passive; active work belongs in the lab

The monitor is metadata-only. It does not download exploit code, scan public targets, or attempt to confirm a vulnerability on production. Those actions require a different authority surface and a different safety envelope.

An item can move toward active validation only when the target is an authorized, disposable replica; the product and version are exact; the detection objective is defined; logging and containment are ready; and reset is proven. The purpose is to test whether a defensive control observes a bounded scenario—not to “see if the exploit works” on a useful system.

1. Build the target from known inputs. Pin the vulnerable component and capture its checksum or image digest.
2. Define one detection objective. Name the event, log, alert, or control expected to fire.
3. Contain every route. Keep the target isolated from public and production networks.
4. Retain the evidence. Record the scenario, observations, timestamps, and limitations.
5. Reset and prove it. Destroy or restore the target, then verify the baseline.

§8What the latest run taught me

The latest seven-day snapshot is useful because the right answer was restraint. Six feed and validation lanes completed successfully. The watcher normalized 11,080 advisories and found 61 inventory-relevant signals. After identity checks, history reconciliation, and evidence gates, 60 remained on watch and zero were promoted for lab review.

Seven apparent alert changes also collapsed into duplicates or non-material differences when compared with the previous successful catalog. Preserving decision history prevented the system from repeatedly interrupting me with the same conclusion.

Primary references

• CISA Known Exploited Vulnerabilities Catalog — confirmed exploitation signal and remediation prioritization input.
• NVD Vulnerability API — CVE records, applicability, metrics, references, and change history.
• FIRST Exploit Prediction Scoring System — probability-oriented exploitation estimate.
• GitHub Advisory Database documentation — ecosystem and package-level advisory search.
• Ubuntu CVE reports — distribution-specific package status and remediation context.

§9Questions I would ask first