Why your engineers ignore PagerDuty (and how to fix it)

The first time an engineer gets paged for something that turns out to be nothing, they shrug it off. The second time, they mute the channel for ten minutes. By the twentieth time, they’ve stopped opening the alert at all — they just wait to see if it resolves itself.

This isn’t a discipline problem. It’s not that your team doesn’t care. It’s that you’ve trained them, one false alarm at a time, to treat your alerts as noise. And once that training is done, it’s very hard to undo.

Alert fatigue isn’t a mood, it’s a learned behavior

Every alerting system teaches its users something, whether you designed it to or not. If 90% of your alerts require no action, your team learns — correctly, rationally — that most alerts require no action. That lesson doesn’t stay isolated to the noisy alerts. It generalizes. The next alert, even a real one, gets the same skeptical half-second glance before someone decides it can probably wait.

This is why alert fatigue is so dangerous: it doesn’t just waste time on false positives. It actively degrades the response to true positives, because your team has learned, through repetition, not to trust the system.

Where the noise actually comes from

A few patterns show up again and again in stacks that have this problem:

Thresholds set by guesswork, not by impact. An alert that fires because CPU crossed 80% tells you nothing about whether users are affected. Thresholds copied from a vendor’s default config or picked because “it seemed reasonable” almost always produce noise, because they’re disconnected from what actually matters — the user experience.

Alerts with no action attached. If an alert fires and the response is “acknowledge it and go back to sleep,” that alert shouldn’t page anyone. It might still be worth logging or reviewing during business hours, but paging is a promise: this needs a human, right now. Breaking that promise repeatedly is what erodes trust the fastest.

Duplicate alerts from the same root cause. One database going down can trigger a dozen downstream alerts — API timeouts, queue backups, dependent service failures — all firing within seconds of each other. Without correlation, your team gets paged twelve times for one incident, and the twelfth page always gets less attention than the first.

Flapping conditions. A metric that hovers right at a threshold triggers alert, resolve, alert, resolve, over and over. Nothing about the underlying system needed twelve notifications, but that’s what got sent.

What fixing it actually looks like

The fix isn’t “add more alerts to catch more things.” It’s usually the opposite.

Start from user impact, not system metrics. Ask what the alert is actually protecting: is a user unable to check out, unable to log in, seeing errors? Build alerting around service-level objectives tied to real user experience, and let the underlying system metrics — CPU, memory, queue depth — feed dashboards for investigation, not pages for interruption.

Give every page a runbook, or don’t page for it. If you can’t write down what a human should do when this alert fires, that’s a sign it shouldn’t be paging anyone. It might still deserve a dashboard, a log entry, or a weekly review — just not a 3am wakeup.

Correlate before you notify. Group alerts by likely root cause before they reach a human. One incident should generate one page with context, not a flood of disconnected symptoms.

Audit alert volume regularly, not once. Pull the last 30 days of pages. For every alert that fired more than once with no real action taken, either fix the underlying threshold or delete it. This is uncomfortable the first time — it usually reveals just how much of your paging volume was never earning its place — but it’s the single highest-leverage thing you can do for your team’s trust in the system.

A Real Case Example

Here’s what this looks like in practice. I was working with a fintech team whose request volume followed a daily cycle — high during trading hours, dropping off at night. A simple threshold alert was either useless or noisy, depending on where they set it. The old threshold fired every night at 2am when traffic naturally dipped, and the on-call engineer had learned to ignore it.

To fix this, I built a dynamic alert that compares the average of requests against the average of errors over the last 24 hours and 7 days. Why those windows? Because 24 hours captures the daily business cycle, and 7 days captures the weekly pattern. The alert only fires when the relationship between traffic and errors deviates from what the business itself defines as normal.

It caught a real API degradation three weeks later that a static threshold would have missed — and it hasn’t fired a single false positive since. The team trusts it now, because the system earned that trust.

The payoff isn’t just fewer pages

Teams that do this work don’t just sleep better, though they do. They also respond faster to the incidents that matter, because every page is credible again. When an alert fires, people move — because the system has earned that response, alert by alert, over time.

That trust is the actual asset. Uptime dashboards and pretty Grafana panels are nice, but an alerting system your team believes in is the thing that actually shortens your incident response time when it counts.