The Real Cost of AI Isn't the Price Tag
The Real Cost of AI Isn’t the Price Tag
Every few months, a new article drops with the same thesis: AI is too expensive, the bubble is about to pop, nobody can sustain these compute bills.
And every time, I think the same thing. These people have never looked at an infrastructure cost curve before.
I’m building CodeGrader, an AI-powered code review tool. I pay for inference. I pay for compute. I watch the bills. The cost is real and I’m not here to pretend otherwise. But I’ve also spent over a decade self-hosting infrastructure, and I’ve watched this exact movie before. The plot doesn’t change. Only the actors do.
The Paper Era
Let me take you back to the 1960s.
If a researcher at one university needed data from another faculty, the process was exactly what you think it was. Someone physically carried paper across campus.
Stacks of documents. Walked down hallways. Handed over at a desk. Maybe logged in a binder if the department was organized enough.
That was the state of the art for knowledge sharing. Slow, expensive, and limited to whoever could physically show up.
We're not talking about printing costs. We're talking about opportunity cost. Research that could have taken days took months. Collaboration between universities meant mailing physical copies and waiting weeks for responses. Entire fields moved at the speed of postal services. The cost wasn't the paper. It was everything that didn't happen because sharing knowledge required a body in a hallway.
The cost of moving information was so high that most information simply didn’t move.
One Cable Changed Everything
Then someone connected two computers with a cable.
A local network. The cost was high, the tech was unproven, and most people in those buildings didn’t see the point. They had paper. Paper worked. Why fix what wasn’t broken?
But once researchers experienced instant data sharing, once they saw a file appear on a screen that was sitting in another building five seconds ago, there was no going back.
That idea didn’t stay in one building. It spread across departments. Then across campuses. Then across cities. Then someone looked at the Atlantic Ocean and said: “What if we ran a cable across that?”
The Insane Bet
Submarine cables. Chip fabrication plants. Global routing infrastructure. Internet exchange points. Data centers the size of warehouses.
Trillions of dollars poured into something most people couldn’t even visualize. If you told someone in 1985 that the world would spend more on undersea cables than most countries spend on defense, they’d call you delusional.
And yet here we are.
You send virtually unlimited data to the other side of the planet. You video-call someone in Tokyo from your couch in Bologna. You deploy a container to a server in Virginia from a terminal on your laptop. Basically for free.
The infrastructure isn't free. Someone pays for the cables, the peering agreements, the power. But the marginal cost per unit of data transferred collapsed so aggressively that for end users and most businesses, bandwidth is effectively a rounding error. That's the pattern. Infrastructure cost doesn't disappear. It gets amortized across so much usage that it becomes invisible.
The return didn’t just justify the cost. It made the cost look trivial in retrospect.
AI Is at the Cable-Laying Phase
AI infrastructure is at the “running cable across the ocean floor” phase right now.
The hardware is expensive. Training runs cost millions. GPU clusters require their own power substations. The compute bills look insane from the outside.
And a whole wave of people are saying the same thing they said about the internet: “This costs too much. It’s not sustainable. Who’s going to pay for this?”
The same people who would have looked at a submarine cable project in the 1990s and called it a waste of money.
The Cost Curve Always Wins
Here’s what every “AI is too expensive” take misses: they’re evaluating a moving target with a still photograph.
The cost is real today. But the return curve on transformative infrastructure always follows the same pattern:
Unbearable at the start
Declining faster than anyone predicted
Invisible once it matures
OpenAI's GPT-4 API pricing dropped roughly 10x within 18 months of launch. Inference costs across the industry are falling quarter over quarter as hardware improves and optimization techniques mature. The same pattern played out with cloud compute (remember when a basic EC2 instance cost $70/month?), storage (S3 pricing has dropped over 90% since launch), and bandwidth. The direction is always the same. The speed surprises everyone.
The question was never “is this expensive?” The question was always “is the return worth the investment before the cost drops?”
With the internet, the answer was so obviously yes that governments funded it. With AI, the market is funding it, which means it’s moving even faster.
The Builder’s Bet
I self-host my entire stack on Hetzner. Kubernetes, Cilium, Longhorn, cert-manager. The whole thing. I’ve done the math on managed services vs. owning the infrastructure, and for my use case, ownership wins.
That same self-hosted stack is what powers ZipOps. Same Hetzner/Cilium setup, zero YAML. If self-hosting isn't the point, just the means, check what we're building.
The same logic applies to AI. I’m paying for inference costs on CodeGrader right now. It’s not cheap. But I’ve watched enough infrastructure cost curves to know that the developers building with AI today, while it’s expensive and messy, are in the same position as those early networked universities.
By the time the cost drops and everyone else shows up, the experience gap is already set. You’ve already learned what works, what fails, what your users actually need from AI-powered features. That knowledge doesn’t come from waiting. It comes from building while it’s still expensive.
The Real Risk
The real cost of AI isn’t what you pay for compute today.
It’s what you lose by deciding you can’t afford to start. It’s the products you don’t build, the workflows you don’t automate, the competitive gap that opens while you’re waiting for prices to drop.
Nobody at those universities went back to carrying paper after they saw what a network could do. Nobody who integrates AI into their development workflow today is going to rip it out when the cost is 10x lower next year.
The trade-off is real You’re betting time and money on infrastructure that’s still maturing. Costs will drop, capabilities will improve, and some of what you build today will need rebuilding tomorrow. That’s the price of being early. But being early is how you end up ahead, not behind.
If you’re watching from the sideline waiting for the “right time” to invest in AI tooling, ask yourself:
Would you have waited for internet costs to drop before connecting your first two computers?
Built on a self-hosted Hetzner/Cilium/Longhorn stack. I handle the infra so I can ship. If you’d rather skip the ops part, ZipOps is the same stack without the YAML.