What breaks: the outage nobody was told about

Three shapes — an alert to A MAILBOX NOBODY READS, a check that PASSED BECAUSE IT MEASURED THE WRONG THING, and a monitor that HAD BEEN DOWN LONGER THAN THE SYSTEM. THE MAILBOX NOBODY READS IS THE COMMONEST AND THE MOST ORDINARY. Alerts go to a shared address, or to somebody who left, or to a folder a rule files automatically. Everything fired correctly and nobody acted, and afterwards the alert history proves the monitoring worked — which makes it worse, because the system is now defended and the outage still happened. THE FIX IS LESSON 5'S NAMED OWNER, AND THE TEST IS TO SEND ONE AND SEE WHO ANSWERS. An alert channel nobody has ever received an alert from is untested infrastructure. THE CHECK THAT PASSED BECAUSE IT MEASURED THE WRONG THING IS THE SUBTLE ONE. A check on a page that does not need the database, so it…

In this module: Module 13 · Monitoring, and knowing before the client does

1. 1What is worth watching
2. 2Checking availability honestly
3. 3The check that runs on the box it watches
4. 4Polling intervals, and what they cannot see
5. 5Alerting somebody who will act
6. 6Alert fatigue as a design failure
7. 7The silent failure: nothing ran
8. 8Reporting to somebody non-technical
9. 9What breaks: the outage nobody was told about