Last updated: September 24, 2026
By Kris Drouet, Engineering Executive, in partnership with KORE1
An on-call rotation that keeps burning people out is usually reporting an architecture problem, because pages cluster at the seams where services are tightly coupled, and adding engineers to the schedule divides that load without reducing it. Total page volume is a design number. Per-person load is the only part headcount touches.
The worst on-call rotation I ever ran was one I designed, and I spent most of a year fixing the wrong thing in it.
Six engineers, weekly shifts, a lending platform that had grown by bolting new things onto old things. People came off their week looking gray. So I did what managers do. I moved the handoff from Monday to Wednesday so nobody lost two weekends in a row, split weekdays from weekends, added a secondary, and eventually got budget for a seventh engineer, who shadowed for a month before taking a week of her own. Every change helped. For about three weeks.
Then two of the original six resigned. Same quarter. On her way out the door, one of them said the quiet part out loud. “It’s not the schedule. It’s the system.”
She had receipts. A spreadsheet nobody had asked her for, holding every page from her last four shifts, each one tagged by where the failure actually started. Most of them started in three places: a synchronous call to the pricing engine, a nightly document sync that fell over and had to be rerun by hand, and a credit vendor’s callback that retried whenever it felt like it. Three seams. I had spent a year rearranging people around a design problem.
I still hear the same request about once a quarter, now from people running rotations of their own. A VP wants two more reqs “for the rotation.” Sometimes they’re right. More often the page log says otherwise, and the page log is the most honest document an engineering org produces, because nobody edits it for the board deck. Reading it takes an afternoon. It belongs with the rest of the unglamorous work I keep defending as operational discipline for engineering teams, the kind that looks optional right up until it costs you people.
What an On-Call Rotation Should Cost
An on-call rotation is the schedule that decides which engineer answers production alerts outside business hours, handed from person to person in fixed shifts, most often a week long. Its real cost isn’t the schedule. It’s how many pages land inside each shift and how many hours each one takes after the alert stops.
Google published the most useful yardsticks I know of in the Being On-Call chapter of its Site Reliability Engineering book, and they hold up outside Google better than most advice from a company that size.
| Guardrail | Google’s number | How I read it |
|---|---|---|
| Share of an engineer’s time spent on call | No more than 25% | Anything past that comes straight out of project work, and it shows up later as roadmap dates nobody can explain |
| Rotation size, one site covering 24/7 | Eight engineers minimum | Below eight, each person’s week comes back before they’ve recovered from the last one |
| Rotation size, two sites splitting the day | Six per site | Twelve people in total, but nobody’s shift runs through the night |
| Work behind each incident | About 6 hours on average | Root cause, remediation, the postmortem, the follow-up bugs; the page itself is the smallest part |
| Incident budget | 2 per 12-hour shift | Past that, the follow-up work stops happening and the same page comes back next week |
Six hours per incident is the figure I watch executives do the math on. Twelve incidents in a week isn’t twelve interruptions. It can be seventy-two hours of work that was never on anybody’s plan, done by the same people you’re counting on for the roadmap.
Most rotations I get asked to look at blow through the last row by Tuesday.
Headcount Divides the Pain Without Shrinking It
Every exhausted rotation has two numbers hiding in it. Leaders almost always look at the wrong one.
Per-person load is how often each engineer’s week comes around and how rough it is when it does. That one responds to headcount. Total page volume is how many pages the system produces, period, and it belongs to the architecture. Headcount doesn’t touch it.
Take a rotation that pages forty times a week. With six engineers, each person holds the pager about nine weeks a year; add two and it drops to six and a half, which is real relief and I’m not going to wave it away. But the week itself still holds forty pages. Forty is forty. Whoever has it still sleeps badly, still spends Thursday on follow-up instead of the roadmap, and still hands the next person a list of things somebody should really fix, which the next person also won’t have time to fix.
The new people aren’t free, either. Somebody has to teach them the system before they can hold the pager alone, and that teaching comes out of the same senior engineers the rotation is already grinding down. It’s the coordination tax that comes with added headcount, paid at night.
Stipends have the same shape. I’m not arguing against paying for on-call. A stipend is a price on the pages, and a price makes a cost easier to accept without making it any smaller. If the stipend line in your budget keeps growing, the architecture is sending finance an invoice.
Pages Live on the Lines, Not in the Boxes
I’ve written before about load-bearing spaghetti, the code nobody touches because everything else quietly leans on it. That piece was about fear inside the codebase. Production has the same spaghetti. The difference is that production doesn’t wait for you to work up the nerve. It calls you.
Here’s the exercise that made this click for me. Print the architecture diagram, the real one and not the one from the investor deck, and put a dot on it for every page from the last quarter, placed where the failure started rather than where the alert fired. The dots never spread evenly. They pile up on the lines between the boxes: the synchronous call that waits on a slow dependency, the database two services both write to, the vendor webhook, the batch job whose output three other things read at 6 a.m.
The Uptime Institute’s Annual Outage Analysis 2026 says the same thing at industry scale. Failures, the May 2026 report found, “are less often the result of a single point of failure and are increasingly linked to complex interactions between systems, including software, networks and external dependencies.” Interactions. Lines, not boxes. The same report counts human error as a contributing factor in nine out of ten outages, which I read less as a verdict on people and more as a description of systems that need a person standing in the gap to hold together.
A page is the architecture telling you where the coupling is, in the only language it has. Usually at 4:10 on a Sunday morning.
When we moved a set of point-to-point integrations onto a Kafka backbone, the number that made the slides was a 45% cut in downstream processing latency. The resilience half of that result is the half an on-call engineer actually feels.
Six Patterns in the Page Log and What Each One Points At
Once the pages are tagged, they sort into a handful of shapes. These six are the ones I run into most, with what’s usually going on under the hood.
| What the log shows | What it usually means | The fix that changes the count |
|---|---|---|
| One failure pages three or four teams inside a minute | Services calling each other synchronously, so one slow dependency becomes everybody’s timeout | A queue or event between them, with timeouts and circuit breakers on the calls that have to stay synchronous |
| The same alert at roughly the same hour, most nights | A batch window colliding with something else, or a job that can’t safely run twice | Make the job idempotent and restartable, then move or split the window |
| Fixed by a restart, again | A memory leak or exhausted connection pool being handled as an operations chore | Count every restart as a defect and put the worst offender on the roadmap |
| Clusters on release days | Big releases with no gradual rollout, so every deploy is a bet | Smaller changes, canary or percentage rollouts, and a rollback that takes minutes |
| Starts at a vendor | An external dependency with nothing standing between it and your core flow | A queue or cache in front of it, plus a degraded mode the business agreed to in advance |
| Acknowledged and closed with nothing done | An alerting problem, not a design problem | Delete it or demote it to a ticket |
Row one fools people. Four teams get paged, four postmortems get written, and each one is careful and correct about its own service. A good incident review will find the local cause every single time. It takes someone reading ninety days of them side by side to notice it was one seam, reported four different ways, and that the fix belongs to none of the four teams. If the seam is a synchronous call sitting inside an otherwise event-driven system, what that one call really costs is worth working out before your next planning cycle.
The nightly page is almost always a retry story. A job dies halfway. Somebody reruns it by hand. Whether that rerun double-posts anything comes down to luck and to whether anyone built for it. I went through the mechanics in idempotency in integration design. The short version is that a job you can’t safely run twice will eventually page somebody for being run twice.
Row three hides in plain sight. The restart works. That’s exactly the problem with it.
Row six is the only one where the fix really is about alerting, and Google’s Rob Ewaschuk set the standard in the monitoring chapter of the SRE book: “Every page should be actionable,” and “If a page merely merits a robotic response, it shouldn’t be a page.” Teams that start there feel better fast. I’d still start there. Just don’t confuse quiet with fixed, because rows one through five are still sitting in the log. They’ve only stopped shouting over the noise.
The Ninety-Day Page Audit
You can run this in an afternoon with a spreadsheet and the engineers who actually carried the pager. Put them at the table. A dashboard can tell you where alerts fired. Only the person who got woken up knows where the failure started.
- Export ninety days of pages from whatever you run, whether that’s PagerDuty, Opsgenie, or incident.io. Pages, not incidents. Deduplication is kind at night and misleading here, because it folds the fan-out from row one into a single tidy line.
- Tag each page with where the failure started. Not the service that alerted. The one that broke.
- Tag how it ended, which is one of five things: a code change, a config change, a restart or rerun, the vendor fixing it, or nothing at all.
- Mark whether a human had to think. That’s Ewaschuk’s test, and it clears out row six in about ten minutes.
- Sort by where the failure started and count. Nearly every time I’ve done this, a short list of seams carries most of the load, and the people around the table could have named them before the spreadsheet did.
- Put hours on it. Google’s six hours per incident works as an upper bound, and even half of that usually makes the case for you.
Then stop. Skip the dashboard. It’s worth rerunning next quarter to see whether the count moved, and it isn’t worth productizing.
One line from the on-call chapter of Google’s SRE workbook belongs taped above every on-call desk I’ve ever seen. “Explaining away a page as ‘transient,’ or taking no action because the system ‘fixed itself’ or the bug inexplicably ‘went away,’ invites the bug to happen again and cause another page, which causes trouble for the next on-call engineer.” Every “transient” in your log is a row you haven’t tagged yet.
Put the Pager Where the Design Decisions Get Made
Google has a rule I’ve borrowed more than once. When a service keeps paging past its budget, the on-call chapter says, “SRE can ask the developer team to be exclusively on-call for the system until it meets the standards.” Hand the pager back.
Amazon arrived at the same place from the other direction. In a 2006 ACM Queue interview, Werner Vogels described the traditional model of throwing software over the wall to operations, then said, “Not at Amazon. You build it, you run it.”
Same mechanism underneath both. The people who can change the design have to feel what the design costs. Hand the pager to a separate operations group that can’t touch the code and you’ve cut a wire. The signal still fires. It just lands on people who can’t change anything about it, so the page count climbs, the ops group burns out on somebody else’s architecture, and the team that built the seam sleeps fine and ships the next one.
That’s an org design question as much as a technical one, and it’s a big part of what an engineering org design and operating model review looks at. If you’re redrawing team boundaries anyway, move the pager with the code. It’s the first change I’d make in any team topology split, before anybody’s title changes.
When It Really Is a Staffing Problem
Sometimes the ask for more people is exactly right. The audit will say so. A four-person team covering all twenty-four hours, seven days a week, sits below Google’s floor of eight no matter how clean the system is; that math doesn’t care about your architecture. A team that has carried a bad rotation for a year is tired, not slow, and it needs relief before it can do any decoupling work at all. And sometimes the audit shows that nobody on the team has ever taken a synchronous tangle apart, which makes it a skills gap rather than a rotation gap.
For that last case the hire isn’t a seventh name on the schedule. It’s one senior engineer who has done the decoupling before and can teach the team while doing it again. That’s usually a site reliability engineer or a senior platform engineer, and sometimes it’s a contract engagement scoped to the decoupling itself, so the permanent team isn’t left carrying both the pager and the rebuild. KORE1’s IT searches close in 17 days on average, which is a lot shorter than most bad rotations last.
Taking the Log Upstairs
Don’t lead with burnout. Executives hear burnout as a morale problem, and morale problems get a pizza lunch.
Lead with hours. Pages multiplied by hours per page is a number a CFO can hold, and it lines up directly against the roadmap dates that slipped last quarter. Then name the seams. Three at most, each with the fix and what it takes. That turns “the rotation is tired” into three funded pieces of work with an owner and a date, which is the same move I make with the tech debt conversation: stop describing the pain and name the thing that’s causing it.
The first thing I say in that meeting is some version of “show me the data.” The page log is the rare case where the data already exists and nobody has read it.
If the hours don’t reconcile with where the roadmap actually went, that gap is its own finding. It’s also the first thing an outside engineering velocity diagnostic goes looking for, because unplanned operational work is where I most often find a quarter that disappeared without anyone deciding it should.
Questions I Get Once the Page Log Is on the Table
How Many Engineers Does an On-Call Rotation Actually Need?
Eight, if one site covers all twenty-four hours, which is the minimum Google’s SRE book sets for a single-site team, and six per site when two sites in different time zones split the day.
Plenty of smaller rotations exist, and the ones that hold up tend to sit on systems that barely page. Size follows volume. Up to a point. The floor is there for the bad month, not the quiet one, so a four-person rotation that feels fine is usually one noisy quarter away from learning why the floor exists.
How Many Pages per Shift Is Too Many?
Google’s own ceiling is two incidents per twelve-hour shift, and its SRE workbook treats going past that as a reason for corrective action, not a reason for the team to toughen up.
Watch the trend. It says more than the count does, because a rotation paging eight times a week and climbing is in worse shape than one paging twelve and falling. The second team is actually fixing things.
Is a One-Week Shift Too Long?
Not by itself, because the length of a shift matters far less than how many pages land inside it and whether the day after is protected for follow-up.
Weekly is the most common pattern for a decent reason: one handoff a week means one context transfer a week. Where it goes wrong is a heavy week with no slack afterward. The follow-up never happens, so the same pages greet the next person. Shortening a brutal week just spreads the brutality around. Fix the week.
Do We Have to Pay Engineers Extra for On-Call?
Short answer: the law often doesn’t require it for salaried engineers waiting at home, and most of the teams I respect pay something anyway.
The Department of Labor’s Fact Sheet #22 on hours worked draws the basic line. An employee required to stay on call on the employer’s premises is working, while one on call at home “is not working (in most cases) while on call.” Hourly and non-exempt staff are a separate conversation, and it’s one for your employment counsel rather than a blog post. The design point still holds. A stipend prices the pages, and if the price keeps climbing, go look at what’s producing them.
Can AI Alert Grouping Fix Our Alert Fatigue?
It will quiet your nights, and it can also hide the one design fact you most need, which is that a single failure is still paging five services.
I like correlation tools at 3 a.m. Rolling forty alerts into one incident is a kindness to whoever is on call. The risk shows up at planning time, when the dashboard says one incident and the architecture says five coupled services. Run the audit on raw pages, not grouped incidents, or the tool will have done its job and yours will still be waiting.
We Can’t Touch the Architecture This Quarter. Where Do We Start?
Start by deleting every alert that paged someone last month and led to no action, because it’s the one fix that needs no architecture budget, and it usually buys back sleep within a couple of weeks.
Then give each on-call engineer the day after their shift for follow-up work and defend that day the way you’d defend a customer meeting. That day is where rows two and three get chipped away one small fix at a time. It’s not a strategy. It’s a floor, and a lot of the teams I walk into are standing below it.
What the Pager Already Knows
Your architecture diagram is aspirational. Your page log isn’t. One was drawn in a planning meeting and the other was written by the system itself, one interruption at a time, and when the two disagree I believe the log.
Before the next req “for the rotation” goes out, pull the ninety days. If the pages are spread thin and the team is simply small, hire. If they pile up on three lines, fix the lines, and bring in someone who has done that work before if nobody on the team has. KORE1 places site reliability, platform, and senior integration engineers on contract and direct hire, and 92% of its placements are still with the client at the twelve-month mark, which matters in this role because most of the value is knowing where your particular seams are. Start the search with a KORE1 recruiter once the audit tells you which kind of person you actually need.
Connect with me on LinkedIn. I’m always glad to compare page logs, and I promise mine from those early years was worse than yours.
