Local AI: you don’t pay twice, you pay three times

A few days ago I came across an excellent piece on tazmenworld, “What hardware, and at what price, for running AI locally in 2026”. If you’re thinking about building an inference rig at home, read it before you read mine: it’s the most honest buying guide I’ve seen all year. It dismantles the “local AI is free” marketing line with a simple, correct idea: you pay twice, once for the machine and once for the watts. Power-draw tables to back it up, a twelve-month electricity bill thrown in for good measure, and a contrarian conclusion I’ll endorse without hesitation: at today’s memory prices, the smartest move isn’t necessarily to buy right now.

I’m not going to redo that work; it’s already done. I want to add the line the spreadsheet leaves out. Because a local machine doesn’t cost you twice. It costs you three times. And the third bill is the only one that isn’t denominated in euros.

What the guide gets right, and where it stops

The article’s reasoning is a clean cost arbitrage: on one side the cloud, which bills you per token; on the other the local setup, which bills you in watts; and in both cases a meter quietly running in the background. The logical conclusion follows: if your usage is moderate, ride it out on APIs for two years, until the memory spike subsides, rather than tying up several thousand euros in a rig bought at the peak.

Economically, it’s airtight. But the arbitrage is drawn across two columns, the price of tokens against the price of watts, and it forgets that choosing “APIs for two years” is not a neutral parenthesis. It’s a commitment. For those two years, you aren’t renting compute; you’re handing your data, your prompts, and soon your agents to a handful of players answerable to a jurisdiction that isn’t yours. That’s a bill too. It just doesn’t land in your inbox at the end of the month.

The third bill: jurisdiction

I laid all this out already in Sovereign AI, so I’ll keep it short. The 2018 Cloud Act lets U.S. authorities compel any American company to hand over data hosted anywhere on the planet, Europe included. OpenAI, Anthropic, Google, and Microsoft are all subject to it, no matter where their servers physically sit. When you “ride it out on APIs,” that’s the contract you’re signing, silently, by default.

Local doesn’t make you more powerful. It makes you sovereign. That’s a property you can’t buy in teraflops and that shows up in no VRAM table. It’s exactly the point I made in AI loops: the agentic loop is sold to you as the great turning point, and it is, but the funnel always ends at a proprietary product to which you hand the keys and the right to act in your name. The cost of tokens, you can see. The cost of sovereignty, nobody bills you, and that’s precisely the problem.

So yes to the article’s arbitrage, but with the third column on the table. “Wait two years” makes accounting sense. It also carries a political price, and that price doesn’t fall when DRAM eases.

The watt has a nationality

The article compares two bills as if a watt were worth a token. Except a watt has a nationality, and so does a token.

The French grid runs at roughly 20 grams of CO₂ per kWh for 2025, according to RTE’s figures. The European average sits around 180. That’s a factor of nine. And a token served from a data center built where the hyperscalers cluster them, Ireland for instance, comes off a far gassier mix than the outlet on your wall in Clermont-Ferrand. Put another way: the watt you burn at home, on nuclear and hydro, is among the cleanest on the continent. The token you offload is not.

That overturns part of the “nothing is free” argument. True, nothing is free; the author’s electricity bill proves it in black and white. But at equal compute, running your GPU in France isn’t merely a transfer from the API bill to the EDF bill. On the carbon side, it’s also the less bad of the two places to spend that energy. I dug into the software side of this in Vibe coding: we tolerate layers of abstraction that burn twenty times too many watts while data centers double their power draw. Sober local compute, on a decarbonized grid, isn’t AI’s environmental problem. If anything, it’s the exception to it.

The memory crunch isn’t weather, it’s eviction

This is where I really want to add my stone, because the article describes the mechanics of the memory spike perfectly but stops at the price. What it describes isn’t a storm you wait out. It’s an eviction.

A reminder of the facts, and they’re harsh. Three manufacturers control more than 95% of the world’s DRAM. They’ve redirected their capacity toward HBM, the memory that feeds the AI accelerators in data centers, because the margins there are far fatter. HBM is expected to absorb about a quarter of DRAM wafer production in 2026, and it’s already sold out for the year. Producing one bit of HBM consumes roughly three times the capacity of a bit of conventional DRAM. Micron flat out shut down its consumer brand, Crucial, to reserve its lines for the server market. The hyperscalers have locked in future supply through multi-year contracts, against a backdrop of some $650 billion in data center capex expected in 2026. And Intel’s CEO sums up what comes next in a single line: no relief before 2028.

Let’s be clear about the engine, because this is where a lot of people get it wrong. It isn’t a conspiracy to deprive you of compute. It’s worse: it’s perfectly rational. Publicly traded companies are facing explosive demand and much higher margins on the server side than on consumer, so they allocate their wafers where the money is. Any executive would do the same. And that diagnosis, far from weakening the worry, makes it sturdier: a conspiracy can be foiled; an incentive structure cannot. As long as servers pay better, consumer memory will stay the adjustment variable. You aren’t the target. You’re the collateral damage, which is the worse news, because no one in that chain has any interest in fixing it for you.

So the advice to “wait two years and use APIs” takes on a particular flavor. Wait for what, exactly? For the oligopoly to spontaneously decide to make less money? For those two years, you’d be funding, token by token, the very infrastructure whose prioritization is squeezing you out of the memory market. I’m not saying the arbitrage is wrong. I’m saying you should see it for what it is: a bet on an easing that nothing in the market’s structure guarantees before 2027 or 2028.

Software claws back what silicon makes pricier

Happily, there’s good news, and the guide misses this one too. While memory gets pricier, the memory you actually need keeps dropping. The VRAM required to run a given model is a moving target, and it’s moving down.

That’s the whole point of quantization and cache compression. I covered it in TurboQuant: you can get down to three bits per value on the KV cache with no retraining, with near-perfect recall over long contexts. Multiply that by the steady progress on the GGUF and llama.cpp side, and the model that demanded 24 GB yesterday demands less tomorrow. A used RTX 3090 therefore ages far better than the memory-price curve would suggest, because the load you’re asking it to carry gets lighter every quarter.

And this is also where real technical sovereignty lives: not in the silicon, but in the open runtime. llama.cpp, vLLM, and Ollama depend on no vendor. The hardware gets pricier, fine, but the layer that drives it can’t be pulled out from under you by a change to the terms of service. It’s the exact opposite of the Blackwell trap the article documents well, where a card sold for AI spends a year and a half poorly supported by the stable stack. The sound sovereign instinct isn’t the flashiest card; it’s the one whose software ecosystem depends on no one.

The limits worth stating

I make the case for local, but not as a blind partisan, so let’s say the uncomfortable parts out loud.

First, local has hidden bills of its own, beyond the watts the article already tallies. A machine that computes makes noise, heats a room, demands maintenance, and ties up capital that sits idle when you aren’t using it. And as much as quantization works wonders, the really big models stay hungry: there’s a VRAM floor no software trick will get under. For light or intermittent use, run the numbers coldly: the API is often the more rational choice, economically and ecologically alike. Owning a machine you switch on three hours a week isn’t sovereignty; it’s waste.

Second, and this is the most uncomfortable limit, running locally doesn’t make you the master of the models. The best open weights, Llama, Qwen, DeepSeek, come overwhelmingly out of American or Chinese labs. You control your hardware, your runtime, and your data, which is already a great deal, but you remain dependent on their decision to publish those weights, and to keep doing so. Local sovereignty stops at the border of the weights. You can push that border back, through fine-tuning, distillation, and support for European models when they’re up to the job, but you can’t erase it. Better to know that before you celebrate.

That said, let’s not push the argument too far, because this dependence has nothing in common with the API’s. Once the weights are on your disk, the cord is cut. No one can hit a button to disconnect you, change the model’s behavior overnight, tighten its alignment, or censor your outputs after the fact. The model may be foreign by birth, but its execution lives at your place, frozen, immune to the silent update and the business decision. You depend on a lab for the next model. You depend on no one for the one already running. That’s the whole difference between renting and owning, and it’s precisely the one local puts on your side.

The real grid isn’t “which GPU”

The article ends on a “which hardware for which need” grid. Very useful. But upstream of it, the grid that decides everything else is missing: which uses justify bringing the work home, and which can stay outside with a clear conscience.

Ask yourself a single question, the one about dependence, not the one about performance. Anything touching client data subject to the Cloud Act, anything that lets an agent act in your name, anything that indexes sensitive material in a RAG: that goes local, whatever the price of memory, because the cost of a sovereignty breach is in no way comparable to that of a used graphics card. The rest, the throwaway draft, the harmless rephrasing, the no-stakes test: leave it on the API; it isn’t worth tying up 3,000 euros. The decision was never “local or cloud.” It’s “what goes local.” And that grid doesn’t read off a column of tokens per second.

My take

The tazmenworld guide is right down the line about what it sets out to cover. You pay for the machine, you pay for the watts, and sometimes the most clear-eyed call is to wait. Go read it; it’s real information, quantified and lived, which is rare.

But the arbitrage it proposes is an accountant’s arbitrage, and the question was never an accounting one. It’s political. The third bill, the jurisdiction one, isn’t settled in euros and won’t fall when DRAM eases. The French watt is among the cleanest on the continent, which flips the ecological argument in favor of sober local compute. And above all, the memory spike isn’t a storm to let blow over: it’s the raw material of your autonomy that the data centers are preempting for two years, while you’re gently advised to sit tight and keep paying by the token.

So if you can, buy used now, a 3090 or a 4090 already in circulation, whose memory was made and paid for at pre-spike prices. Not out of nostalgia for tinkering. Because in a world where memory goes first to wherever the margin is fattest, which is to say far from you, owning your own is already an act of resistance.