Retry: how many, how far apart

IMMEDIATE RETRIES HELP WITH NOTHING. Whatever made the call fail a millisecond ago is almost certainly still true, so an immediate retry is a second failure at no benefit — and three of them is a third. BACKOFF is the fix: wait, and wait longer each time. A few seconds, then tens, then minutes. That gives a restarting service time to come back and a busy one time to clear, which is the entire population of failures worth retrying. A LIMIT IS NOT OPTIONAL. Retrying forever turns a permanent failure into permanent load, and the flow never reaches the handler that would have told somebody. Three to five attempts covers the transient cases; beyond that you are retrying something that needs a person. THE RETRY STORM IS THE FAILURE THAT TURNS A SLOW PARTNER INTO AN OUTAGE. The target slows down; calls start timing out; every one is retried immediately;…

In this module: Module 8 · Error handling, faults and retries

1. 1The four kinds of failure
2. 2Faults versus errorsFree preview
3. 3Scope and fault handlers
4. 4What to do with a caught fault
5. 5Retry: how many, how far apart
6. 6The dead letter, and who reads it
7. 7Partial failure in a batch
8. 8Compensation, and its limits
9. 9Alerting a human
10. 10Resubmission: doing it safely
11. 11Errors that are not failures
12. 12What breaks: five error-handling failures
13. 13Lab briefing · Break it five ways