About this article
This article is the System Architecture category’s final installment (11th) in the Architecture Crash Course for the Generative-AI Era series, covering cost management (FinOps) in cloud.
Cloud’s pay-as-you-go can be $1k or $100k a month depending on one decision. The article covers what makes bills balloon, the three phases of FinOps (continuous optimization of cloud-cost operation), pricing models, tag strategy, management levels by scale, and the LLM-era new cost lenses.
What is FinOps (cost management) in the first place
FinOps is, roughly speaking, “the ongoing activity of visualizing cloud usage costs and eliminating waste.”
Imagine your water bill. Leave a faucet running and the bill balloons; fail to shut off water to rooms nobody uses and it keeps flowing needlessly. Cloud is pay-as-you-go too, so instances left running, excessive logging, and abandoned resources routinely produce bills 3-10x the estimate. FinOps isn’t mere cost-cutting — it’s investment optimization with the framing “pay where it matters, cut waste.”
The day your finger trembles opening the invoice
The umbrella term for “continuously visualizing and optimizing cloud usage cost” is FinOps. Not mere savings — investment optimization with the framing “pay where it matters, cut waste.” Cheap doesn’t mean win; whether spend matches business value is what FinOps keeps asking.
Bills hitting 3-10x the estimate are not unusual: too many logs, instances left running, “behind-the-scenes calls” of managed services. Most of these are preventable at design time; “didn’t read the pricing model at design” is the simple omission that causes them.
Cost is a non-functional requirement that hits monthly once operations start. Defending at design time is the rule.
Why think about it first
1. 3-10x billing accidents are common
Too much logging, idle instances, managed-service “behind-the-scenes calls” — bills coming in 3-10x the estimate are recurring. LLM APIs especially scale to thousands of dollars monthly fast; forgetting prompt caching alone spikes the bill.
2. Retroactive fixes are expensive
Reorganizing the architecture to optimize cost is close to a One-way Door. Without thinking at initial design, you regret it. “Think about it after launch” almost always produces months of wasted spending.
3. Pair design with approval authority
Cost is an executive-decision domain. Up to $1k a month: field decision. $10k+: executive approval. “Spending governance” must be agreed at design time. Leaving this vague at launch produces fights between executive and field at month-end.
”Common pitfalls” that explode bills
The typical causes of bill explosions repeat the same patterns surprisingly often. They share “could have been caught at design time.”
| Pitfall | What happens |
|---|---|
| Forgetting to stop dev environments | Charged 24h including nights and weekends — hundreds of dollars/month |
| Excessive log output | CloudWatch Logs cost exceeds production-app cost |
| Communication via NAT Gateway | Data transfer costs balloon unexpectedly |
| Over-spec managed DB | ”Just in case bigger” can be 10x the cost |
| S3 old objects left | Without deletion policy, charges accumulate |
| Data egress | Cross-region / cross-AZ — hard to see |
Egress (traffic leaving the cloud) is the most overlooked billing item.
FinOps in 3 phases
FinOps spins through “Inform (visualize) -> Optimize -> Operate (operationalize).” Many companies skip even “visualize” and jump to savings tactics that don’t stick.
| Phase | Substance | Tools |
|---|---|---|
| Inform | Visualize who / what / how much | AWS Cost Explorer, GCP Billing Reports, Azure Cost Management |
| Optimize | Delete unused, right-size, apply discounts | AWS Compute Optimizer, Trusted Advisor |
| Operate | Governance, budget caps, continuous improvement organization | AWS Budgets, tag conventions, weekly reviews |
Without going “visualize -> optimize -> operationalize” in order, short-term savings don’t stick and revert. Skipping the Operate phase (the org-level mechanism for keeping it going) leaves it as a one-time event.
Major pricing models
Cloud’s base pricing is pay-as-you-go, but long-term commitments offer significant discounts. Understanding models is mandatory because choices can halve the bill.
| Model | Substance | Discount |
|---|---|---|
| On-demand | Pay as you use | Baseline (list price) |
| Reserved Instances (RI) | 1-3 year commit | Up to 70% off |
| Savings Plans | Usage commit (AWS) | Up to 70% off |
| Spot / Preemptible | Interruptible / spare capacity | Up to 90% off |
| Committed Use Discounts | Long-term discount (GCP) | Up to 57% off |
Reserved for steady-load production, Savings Plans for variable load, Spot for batch — the standard pattern. RI is loss if unused, so coverage and utilization monitoring is part of operations.
Visualization basics: tag strategy
The premise for cost visualization is “a tagging convention covering all resources.” Untagged resources show up as “unknown cost” on the invoice; nobody knows who used them or why.
| Required tag | Examples |
|---|---|
| Environment | prod / staging / dev |
| Project | project-alpha |
| Owner | team-a / foo@example.com |
| CostCenter | Department code |
| Service | web / api / batch |
“A mechanism to enforce tagging” (auto-stop untagged resources, etc.) combined with IaC is the modern standard. “Tag it later” never happens.
Decide the tag convention at the start of design. Bolting on after launch is hell.
Decision criteria — scale and phase
The strictness of cost management changes by monthly spend and business phase.
| Phase | Monthly target | Management level |
|---|---|---|
| MVP / validation | up to $1k | Budget alerts only |
| Early growth | $1k-10k | Tags + monthly review |
| Scale | $10k-100k | Dedicated FinOps, RI/SP usage |
| Enterprise | $100k+ | Specialized team, automated optimization |
Excessive optimization at MVP is a waste of time, but past around $5k/month, “running cost reviews weekly” starts paying off.
Costs decided by architecture characteristics
Architecture selection itself decisively impacts cost.
| Design decision | Cost impact |
|---|---|
| Going serverless | Free when idle; effective at low/mid traffic |
| Containerization | Higher density; multiple services on one machine |
| Managed-service dependence | Operational cost down, unit cost higher; watch egress |
| Multi-region | Availability up, but cost premise is double |
| Data transfer volume | Cross-region, cross-AZ, CDN — be aware |
“Serverless vs container” especially produces 2x+ differences depending on request patterns; evaluate carefully.
Selection by case
Startup MVP
Set a monthly cap alert via AWS Budgets. Tags can be just Environment / Owner. Default to serverless (Lambda, Vercel, Cloud Run) at zero fixed cost; revisit when growing. Supabase / Neon free tiers are more rational than managed RDBs.
Mid-sized B2B SaaS
Monthly review via Cost Explorer, mandate 5 tag fields, get 30-50% off via RI / Savings Plans. Shorten CloudWatch Logs retention to 7 days, auto-stop unused dev environments at night. At this scale, manual ops hits a ceiling and writing automation scripts becomes worth it.
Large enterprise
Dedicated FinOps team, AWS Organizations for per-department account separation, auto-detect / auto-stop untagged resources. Apply AWS Compute Optimizer recommendations monthly, maintain RI coverage at 80%+. Multi-cloud management tools (CloudHealth, Apptio) come into view.
Data platforms / AI utilization
BigQuery bills by query volume — “SELECT * is fatal.” Snowflake’s warehouse size bites directly. Working columnar with explicit columns, partition pruning, query caching is high-impact.
LLM usage scales to monthly tens of thousands of dollars instantly via tokens × requests. Prompt caching can cut input cost 10x; model selection (Haiku → Sonnet → Opus) is 10x. Per-user upper bounds and abuse prevention are mandatory.
FinOps operational traps
Cost is the area where “don’t do anything and your bill doubles by the time you notice.” 90% gets decided by the first operational rules.
| Forbidden move | Why |
|---|---|
| Decide tag conventions after creating resources | Retroactive tagging never finishes; unknown cost exceeds 50% |
| Buy RI / Savings Plans without coverage monitoring | ”Dead-bought” expires unused — 1-3 year commitments lock in losses |
| Don’t auto-stop dev, run 24/7 | Sum of dev/staging frequently exceeds production |
| DEBUG logs in production | CloudWatch Logs bills $10k+/month — filter to INFO+ and shorten retention |
| Adding NAT GWs unconsciously | One = $30-60/month + traffic; charges accumulate even in unused environments |
| No S3 lifecycle policies | Years-old videos / images charged at Standard class indefinitely |
| Multi-region without estimating egress | Cross-cloud / cross-region / cross-AZ traffic comes 10x the estimate |
| First notice of cost anomaly is the month-end invoice | Loss already locked in for the month — set up AWS Cost Anomaly Detection |
| Provide LLM API without per-user cap | Malicious users’ prompt loops can hit $10k overnight |
| Adopt managed services without reading the back-side billing | The worst contracts come from not reading the docs. Weekly invoice review is the only defense |
| Designing with the premise of starting cost optimization after launch | Architecture can’t be changed later. Not folding pricing models in at design locks in months of wasted spending |
| Evaluating managed services by unit price alone without calculating TCO | Unit price is higher but TCO including operational labor and incident response is often lower. Decide including labor costs |
Even Amazon internally publicized that the Prime Video team in 2023 went back from microservices to monolith and cut cost by 90%. Architecture selection drives cost; not reading pricing models at design time is fatal.
Cost incidents come from “forgot to stop.” Defend with auto-stop, not human discipline.
AI decision axes
With AI-driven development and AI utilization as the assumption, cost management moves from “humans optimize” to “AI suggests and auto-applies continuously.” AWS Cost Anomaly Detection and Compute Optimizer are already AI-based, and “anomaly detect -> root cause -> recommend -> auto-apply” is becoming standard.
| AI-era favorable | AI-era unfavorable |
|---|---|
| Tag conventions and IaC | Manual provisioning |
| Standard service configurations | Unique exotic configurations |
| Auto-collected usage metrics | Monitoring not in place |
| LLM token management | Unlimited prompts |
Meanwhile, AI’s own usage cost becomes a new management target. LLM APIs scale to tens of thousands of dollars monthly fast; prompt caching, model selection, and per-user upper bounds are the new pillars of FinOps.
- Tag convention and visualization first — without sight, nothing improves.
- Pick pricing models at design time — retroactive fixes are huge cost.
- Management level matched to scale — dedicated FinOps for MVP is excess; mandatory at scale.
- AI usage cost as the new central axis — LLM and AI-agent spending become the focus.
LLM API cost management is the new core of FinOps
As of 2026, LLM API usage costs are ballooning to match or exceed infrastructure costs. Claude/GPT-4 class models cost several to 15 dollars per million input tokens, with output at 3-5x that rate. Per-user-request cost reaches a few cents, so cost projections under traffic growth follow different math than traditional compute billing.
Three effective countermeasures: prompt caching (reusing responses for identical inputs), model routing (Haiku/small models for simple tasks, Opus/large models for complex tasks), and per-user usage caps.
AI-driven auto-recommendation for cost optimization
AWS Cost Anomaly Detection, Compute Optimizer, and Trusted Advisor already provide AI-based recommendations like “this EC2 instance has been below 5% CPU for 30 days — downsize it.”
With IaC-managed infrastructure, generating these recommendations as Terraform change PRs is also buildable. An operational flow where AI handles “anomaly detection -> root cause -> fix PR generation -> human approval” end-to-end is becoming standard.
”Year-end / new-year billing” (industry case)
A team rushing through year-end release work spun up GPU instances for verification, planned to “stop them later,” and went into the new-year break. On the first day back, the cost dashboard showed the monthly bill at over 10x normal, with the culprits being those forgotten GPU instances.
Personal-developer episodes exist: a verification r5.2xlarge left running over the weekend melted hundreds of dollars. Hundreds is funny; tens of thousands on GPUs isn’t.
The lesson: “‘Stop it later’ depends on human will, and almost always breaks down.” Embed mandatory auto-stop scripts at night/weekends with tags from day one. Defend cost with design, not willpower.
Don’t scold forgotten stops. Tilt the system to stop on its own.
What you must decide — what’s your project’s answer?
Articulate your project’s answer in 1-2 sentences for each:
- Tag convention (Environment / Project / Owner / CostCenter)
- Budget caps and alerts (monthly / weekly)
- Pricing-model selection (on-demand / RI / Savings Plans)
- Dev-environment auto-stop policy
- Log retention and data-deletion policy
- Cost-review cadence (weekly / monthly)
- LLM / AI-service usage caps
Related Articles
https://en.senkohome.com/arch-intro-system-overview/ https://en.senkohome.com/arch-intro-index-system/ https://en.senkohome.com/arch-intro-system-application-types/
Summary
This article covered cloud cost management (FinOps) — the patterns of bill explosion, 3-phase operation, tag strategy, management levels by scale, LLM-era topics.
Set the tag convention first, fold pricing models into design at design time, operationalize by scale. In this order, you avoid the “open the invoice and turn pale” incident.
This concludes the System Architecture category’s 11 articles. The next article opens the Software Architecture category — monolith vs microservices, language selection, API design, and other selection axes for software’s internal structure.
Back to series TOC -> ‘Architecture Crash Course for the Generative-AI Era’: How to Read This Book
I hope you’ll read the next article as well.
📚 Series: Architecture Crash Course for the Generative-AI Era (16/89)