Noisy On-Call Rotations
Noisy on call rotations with a lot of false positive alerts are doubly dangerous.
They ruin the morale of the engineering team and they drive engineers to ignore them, not take action on them and resolve them without properly following up or investigating.
Once you get in this state, it is crucial to exit it as soon as possible.
First, consider turning off paging alerts until you are in a healthier state. Second, it is important to dedicate engineering time to get the rotation into a healthy state with a streak of days without unnecessary alerts.
Software Engineering from the Frontlines Course on Maven
If you liked this article, I will be teaching a “Software Engineering from the Frontlines” course on Maven where I will teach hard-learned lessons I acquired developing large-scale products at companies such as Uber, Airbnb, and Microsoft.
