Cloud Cost Optimization Is an Architecture Problem: Where the Money Actually Goes

Last updated: September 11, 2026

By Kris Drouet, Engineering Executive, in partnership with KORE1

The mandate arrived as one line in a planning doc. Reduce cloud spend 20% by the end of Q3. No context. No owner below the VP. Just a FinOps platform, already purchased, waiting for somebody to log in.

The team did everything the platform recommended, and did it well. Savings Plans against the steady baseline. A few hundred instances rightsized, most of them provisioned by an engineer who had left two years earlier. Dev and QA scheduled to sleep at night. Eleven percent in six weeks. Everybody got a nice email.

Then it stopped.

Month seven looked like month six. Month eight was worse. When I finally sat down with the bill sorted by usage type instead of by service, the biggest line that had not moved was not compute at all. It was a NAT gateway. Services in private subnets were pulling scanned loan documents out of S3 through that gateway because nobody had ever added the free endpoint, so every page of every loan file paid a processing charge on the way in. Then the same image was copied to an OCR service in a different Availability Zone and paid again on the way across. Under the hood, the whole line was one missing route, repeated a few thousand times an hour.

No tool flagged it. Technically it was not waste. The system worked as designed. The design was the leak.

That is the part of cloud cost nobody sells you a dashboard for. A rightsizing tool can tell you an instance is too big. It cannot tell you an arrow on your architecture diagram is expensive. Whiteboard arrows are free. In production, every one of them carries a price per gigabyte.

If you were handed the same mandate and you are building the budget case around it, I have written separately about building an engineering budget your CFO will approve. This piece is about the bill itself. I write these in partnership with KORE1, whose IT staffing practice places the cloud architects and platform engineers I am about to describe, so weigh my opinions about hiring accordingly.

Two Different Jobs Hiding Under One Name

Cloud cost optimization is the work of lowering what you pay to run cloud workloads without giving up the performance or reliability the business depends on. It splits into two jobs. Rate optimization lowers the price of each unit you consume. Architecture optimization lowers how many units your design consumes in the first place.

Nearly every tool on the market is built for the first job, and so is nearly every article ranking for the phrase. Commitments. Reservations. Rightsizing. Spot capacity. Deleting the unattached volume somebody forgot about in 2022. All real, all worth doing, and I have never told a team to skip them.

They share a ceiling, though. The ceiling is the design. A 30% discount on a service that drags the same document across three Availability Zones is still a service that drags the same document across three Availability Zones. You pay less per trip.

People who run cost programs full time are seeing this in their own numbers. The FinOps Foundation’s State of FinOps 2026 surveyed 1,192 practitioners managing more than $83 billion in annual cloud spend, and workload optimization and waste reduction remain their single top priority. The quotes underneath that headline are more interesting than the headline. One practitioner said, “We have hit the ‘big rocks’ of waste and now face a high volume of smaller opportunities that require more effort to capture.” Another described reaching 97% optimization in their Cost Optimization Hub.

I believe both of them. I also think they are describing the edge of what a cost tool can see. Ninety-seven percent measures how much of the tool’s own advice you have taken, and it says nothing about the arrows. The rocks are still there. They moved into the design, where a cost team can see them and cannot touch them.

Where the Money Goes Before Anyone Opens the Bill

Most of the cloud bills I get asked to explain are not a pricing problem. They are a design problem that nobody wrote down. The lines that grow fastest nearly always trace back to one of four decisions made in a design review, or more often in the hallway right after one.

Data That Moves Because Nobody Priced the Arrow

I look here first. Engineers underestimate this line more than any other on the bill. On AWS, traffic between Availability Zones in the same region costs $0.01 per gigabyte in each direction, which means a service in one zone talking to a database in another gets charged twice for the same conversation. Both ends pay. Data leaving AWS for the internet starts at $0.09 a gigabyte once you pass the first 100 GB in a month. A NAT gateway charges $0.045 for every gigabyte it processes, plus $0.045 an hour just for existing, while the S3 and DynamoDB gateway endpoints that keep that traffic off the NAT entirely carry no additional charge. None of those numbers looks alarming alone.

A penny sounds like nothing. Multiply it by a replication stream.

The table below prices a few defaults I run into constantly, at list price, so you can see the scale before anyone argues about it.

Your negotiated rates will be different. The ratios will not be. Every one of those rows is a decision, not a price.

Capacity Sized for a Day That Comes Twice a Year

Somebody sized the cluster for Black Friday. Or for month-end close, or for the afternoon a partner integration replayed three days of events at once. The peak passed. The capacity stayed.

Autoscaling was supposed to fix this, and it helps, but only for workloads designed to scale out and back in, which in practice means stateless services, short-lived connections, and a startup time measured in seconds rather than minutes. A stateful service that needs nine minutes to warm its cache does not autoscale. It gets pinned at peak by an engineer who was paged for it once and has no intention of being paged for it again. I have been that engineer.

Non-production is the same problem with less excuse. A QA environment that runs 168 hours a week and gets used for about 50 is paying for 118 hours of nothing, roughly 70% of its compute. The fix is not a tool. It is a default.

GPU capacity held for an inference peak is the 2026 version of this, and it is the line I watch most closely on any bill with AI in production. The FinOps Foundation found 98% of practitioners now manage AI spend, up from 31% in 2024, and many organizations are being asked to fund new AI work out of optimization savings. That raises the stakes. I went through what the AI line does after launch in what an AI pilot costs once it is running in production.

Retention Nobody Chose

Nobody in the history of software has walked into a design review and proposed keeping every debug log forever. And yet.

By default, CloudWatch Logs stores log data indefinitely unless somebody sets a retention policy on the log group, and ingestion runs $0.50 per gigabyte before a single byte is stored. So a team that turned on DEBUG logging in production to chase one incident, and never turned it off, pays to ingest all of it every day and then pays every month to keep it. EBS snapshots with no lifecycle policy work the same way. So does versioning left on for an S3 bucket that gets rewritten every hour.

Each of those was a reasonable call in the moment. None was ever revisited.

This is the clarity problem I keep writing about, showing up as a line item. When I run the Clarity Stack on an engineering org, the third layer always comes back to decisions that happened in hallways instead of in systems. Retention is nearly always one of them. Nobody decided it. So nobody owns it.

Service Boundaries Drawn Around the Org Chart

The most expensive architecture decisions I see have nothing to do with instance families. They are about where one service ends and the next one begins.

Split a workflow into ten services and every boundary becomes a network hop, a serialization step, a retry policy, a queue, and often a trip across a zone. That can be exactly right. It can also be a very expensive way to draw an org chart. I have priced what one synchronous call costs inside an event-driven system before, and the dollar side of that argument is usually bigger than the latency side.

The cleanest public example is still Amazon’s own. Prime Video’s video quality monitoring team built its first version on AWS Step Functions and Lambda, passing video frames between components through S3. According to InfoQ’s 2023 write-up, that design supported only around 5% of the expected load, and the orchestration hit account limits because it ran several state transitions for every second of every stream. The team pulled the pipeline into a single application on ECS, moved the frame handoff into memory, and cut operational costs by 90%. Same company. Same cloud. Different boundaries.

I do not read that as an argument against microservices. I read it as an argument for pricing a boundary before you draw it.

Managed services deserve a short note too. Paying a premium for a managed database or queue is often the right build vs buy call, because you are buying operations work you would otherwise have to staff, and my framework for deciding what to build and what to buy treats that operating cost as the number people most often leave out. The mistake is paying for resilience the workload never needed, like a multi-zone managed cluster behind an internal reporting job. That job could be down all afternoon. Nobody would notice.

Why the Savings Stall in the Same Place

Back to the eleven percent.

That team’s savings stalled for a reason that had nothing to do with effort. The FinOps analyst could see the NAT line and had no authority over the subnet design. The engineers had full authority over the subnet design and never saw the NAT line, because the bill went to finance and the only cost number engineering ever heard was the total, once a quarter, delivered as a complaint.

The people who could see the problem could not fix it. The people who could fix it could not see it. That is an ownership gap. Not a cost problem.

The State of FinOps data points at the same gap from the other side. Pre-deployment architecture costing emerged as a top desired tool capability, which is a polite way of saying practitioners want the number before the design ships instead of eighteen months after. FinOps teams are already sitting down with platform engineering and enterprise architecture groups, building pricing calculators and offering pre-deployment guidance, but in the report’s own words, “incentive structures haven’t caught up.” One practitioner put it better than I can. “Once you fix it, it’s gone. How do we give developers credit for shift-left activities?”

That question deserves more airtime than it gets. Avoided cost is invisible. An engineer who designs the NAT gateway out of a system before launch saves real money and has nothing on any dashboard to show for it, while the engineer who finds the same problem a year and a half later gets a slide in the quarterly review. Guess which behavior the org learns to reward.

Make Cost a Design Requirement, Not a Monthly Surprise

Werner Vogels opened his re:Invent 2023 keynote with a set of rules he called The Frugal Architect. Law I is “Make Cost a Non-functional Requirement.” Law IV is “Unobserved Systems Lead to Unknown Costs.” When Amazon’s own CTO says cost belongs in the architecture, it is a little strange that most engineering orgs still treat it as something accounting finds out about later.

I agree with him. I would add the part that keeps getting skipped. A requirement with no owner is a suggestion. Somebody has to hold cost in the design review, the same way somebody holds security and somebody holds latency.

These are the questions I now ask before anything with real volume ships:

• What moves, and how far? Every arrow on the diagram gets a rough monthly gigabyte figure and a note on whether it crosses a zone, a region, or the internet.
• What runs when nobody is using it?
• How long do we keep what this writes, and who decided that? If the answer is “the default,” that is the answer I push on.
• Could this boundary be a function call instead of a network call? Often it cannot. I still want to hear why.
• What does one unit of business, a loan file or a tenant or an order, cost to run on this design today, and what does it cost at ten times the volume?

Five questions. Ten minutes. The first time a team hears them, it feels like overhead. By the third review they are answering before I ask.

The Two-Week Diagnosis

If you already have the mandate and not the luxury of a redesign, this is the order I work in. It takes about two weeks with one engineer who knows the system and one person who can read a bill.

1. Pull twelve months of spend from Cost Explorer or the Cost and Usage Report, grouped by usage type rather than by service. “EC2” is not a diagnosis. DataTransfer-Regional-Bytes is.
2. Take the top twenty lines and label each one rate, hygiene, or design. Rate lines get commitments. Hygiene lines get deleted or scheduled. Design lines go to step three.
3. For every design line, find the decision behind it. Which arrow, which default, which boundary, and who made the call. This is not about blame. You are finding out whether anyone still owns it.
4. Price each fix the way you would price a feature, in engineering weeks, risk, and monthly savings, so it can be ranked against roadmap work instead of sitting in a backlog with a tech debt label on it. I have argued that “tech debt” is the wrong frame for exactly this kind of request, and cost is where that argument pays off fastest.
5. Ship the cheapest structural fix first and put the result in front of whoever issued the mandate.

Step five matters more than it looks. The first structural fix buys you permission for the next four, but only if somebody outside engineering watches the number move.

Who Owns the Design Lines

A FinOps function is good at finding. It is rarely staffed to fix. The fix belongs to whoever owns the architecture, and at a lot of mid-market companies that seat is either empty or held by a principal engineer who is also the on-call escalation for everything.

Two roles close the gap in the organizations I have seen do this well.

The first is a cloud architect who has personally owned a bill, not just a diagram. You can hear the difference in an interview. Ask what the most expensive arrow they ever drew cost per month, and whether they caught it before or after it shipped. People who have lived it answer in dollars. People who have not answer in principles. That is the first question I would want asked in any cloud architect staffing search.

The second is a platform team that makes the cheap path the default path. Retention set in the template. Gateway endpoints in the base network module. Non-production schedules switched on unless someone opts out. Nobody has to remember, because nobody has to choose, and that is most of what platform engineering staffing is really buying you, whether or not the job description says so.

KORE1’s placements hold at 92% after twelve months, and for an architect that number matters more than for almost any other hire. When the person who drew the arrows leaves, the reasons behind them leave too, and whoever comes next inherits a bill they cannot explain. If you need the diagnosis before you can justify a permanent seat, a contract architect on a fixed ninety-day engagement is usually the easier approval and a cleaner test.

Before You Take the Mandate Back to Finance

Which of the “Pillars” of Cloud Cost Optimization Should We Start With?

Visibility, because you cannot redesign a line you cannot trace back to a decision. Every other practice area depends on knowing which design produced which cost.

The pillars people quote vary depending on who is selling what. AWS’s own Well-Architected cost optimization pillar lists five practice areas, which are cloud financial management, expenditure and usage awareness, cost-effective resources, managing demand and supply, and optimizing over time. Fine list. Awareness is the one that decides whether the other four ever get past the rate layer, so tag spend by owner and by product before you try anything clever.

Can We Hit a 20% Cut This Quarter Without Touching the Architecture?

Sometimes, but only if you have never bought commitments or scheduled non-production environments, since those two can carry a first-year number with no design change. After that, the easy money is gone.

Add retention policies on every log group and gateway endpoints for S3 and DynamoDB, and you have the no-regret list. Do all four. Then expect year two to be harder, because the discount already happened and the design did not change. The leaders who promise another 20% the following year based on how easy the first one was are the ones who end up in my inbox.

FinOps Analyst or Cloud Architect, Which Seat Comes First?

The architect, if spend is already visible by owner. An analyst tells you which lines are growing, but only someone with authority over the design can make them stop.

If you cannot attribute spend to teams at all yet, flip the order. Hire or contract the analyst first, get tagging and allocation working, and then bring in architecture help with a real target list. The two seats are priced very differently, so check current cloud architect salary data before you write the requisition.

Would Moving Workloads Back On-Premises Fix This?

Repatriation moves the bill, but it rarely fixes the design.

Chatty services and unbounded retention are expensive in your own data center too. They just show up as storage arrays, network gear, and headcount instead of line items with a usage type attached. Repatriation can be the right call for steady, predictable workloads sitting next to a lot of data. It is the wrong call when the real problem is a design nobody has priced, because you will carry that design into a building where it is even harder to see.

How Do I Get Engineers to Care About a Bill They Never See?

Engineers care about what shows up in their design review, so the cost has to show up there, in dollars per month, before anything ships.

Two changes do most of the work. A rough monthly cost estimate goes into the design doc template next to latency and availability, and it gets recorded. Then, when a design ships at or under its estimate, somebody says so out loud in the same meeting where features get celebrated. That is how avoided cost finally earns credit. Nobody objects to this. Almost nobody does it.

Are Cloud Cost Optimization Tools and Services Worth Paying For?

Short answer: yes for the rate layer, no for the architecture. Cloud cost optimization tools are good at commitments, rightsizing, and finding idle resources, and weak at the design decisions that set your floor.

Buy them with that split in mind and they earn their fee. The disappointment comes when a leadership team buys a tool expecting an architecture outcome. A tool can see that a service costs money. It cannot see why the service exists.

Read the Bill Like a Design Document

Pull up last month’s bill. Not the total. The top twenty usage types.

For each one, ask a single question. Is this line a price, or is it a decision? Prices you can negotiate. Decisions you have to revisit, with the person who can change them in the room and a diagram open that nobody has looked at since the day it was drawn.

If you run that exercise and want a second read on what you find, message me on LinkedIn with your top five lines, and I will tell you which ones I think are design. And if the diagram behind those lines has no living owner, talk to a KORE1 recruiter about the seat before you spend another quarter chasing discounts.