A 1,000x Power Cut Is an Engineering Problem, Not a Press Release

On June 25, 2026, TechCrunch reported that Naveen Rao — formerly head of AI at Databricks — is running a startup called Unconventional AI with a striking claim: an oscillator-based computer architecture that could cut AI inference power by as much as 1,000 times. The company released a first model, Un-0, an image generator that performs comparably to Stable Diffusion and GPT Image 1 — but runs on a software simulation of the oscillator chips, not the hardware itself. Rao’s own framing is honest: it’s “the ‘hello world’ of a new kind of computer,” and “much of the infrastructure to get there is still being built.”

So treat the 1,000x number for what it is — a research target on a simulator, not a product you can buy. The part worth your attention is the diagnosis underneath it, which Rao states plainly: “AI scaling is hard because of energy,” and power availability will be “the fundamental limit in the next few years.” He’s right. And waiting for exotic silicon is the wrong response, because most teams are leaving large efficiency wins on the table with the hardware they already have. AI energy efficiency is an engineering problem you can work on this quarter — not a press release you wait on.

Where the power actually goes#

Before optimizing anything, you have to know where the joules are spent. There are three buckets, and most conversations conflate them.

Training is a fixed, amortizable cost. Pre-training a frontier model is enormously energy-intensive, but it happens once and serves billions of requests. For nearly every team that builds on models rather than trains them, training power is somebody else’s bill. Optimizing it is not your lever.

Inference is the recurring bill, and it scales with usage. Every request you serve burns energy, forever, for as long as the feature lives. A model that costs a little more per call but is invoked millions of times a day dwarfs a one-time training run in total lifetime energy. This is where your decisions actually move the needle, and it’s exactly the workload Rao’s architecture targets — inference, not training.

Data movement is the hidden tax on both. This is the part teams miss. A surprising share of the energy in modern accelerators goes not to computation but to moving data — shuttling weights and activations between memory and compute units. Fetching operands from off-chip memory costs orders of magnitude more energy than the arithmetic that consumes them. That’s why architectural bets — Rao’s oscillators, the custom silicon wave we covered earlier, and the broader industry push — keep circling back to the memory wall rather than raw FLOPS. You can’t fix the physics of someone else’s chip, but you can stop moving data you don’t need to move.

Latency and cost budgets are data problems#

Here’s the reframe that makes efficiency tractable: your latency budget and your cost budget are the same budget, and both are dominated by how much data you push through the model. Energy, latency, and dollars move together — shrink the data flowing through inference and all three drop at once. That turns a vague “be greener” goal into concrete data-engineering work.

Send fewer tokens. Token count is the most direct dial you have. Bloated system prompts, redundant context, and entire documents stuffed into a window when a paragraph would do — each token is compute, latency, and energy. Tightening prompts and retrieving only the relevant chunks is the cheapest efficiency win available, and most teams have never measured how many tokens they actually need.

Don’t call the big model when a small one answers. Capability-based routing — a small fast model for easy requests, the heavyweight only when the task demands it — can cut energy per request by a large factor with no quality loss on the easy path. The discipline is measuring which requests genuinely need the big model rather than defaulting everything to it.

Cache aggressively. Identical or near-identical requests should never hit the model twice. Response caching for repeated queries and prompt-prefix caching for shared context both convert a full inference into a near-free lookup. A cache hit is the most energy-efficient inference there is — it’s the one you didn’t run.

Batch and right-size. Batching requests improves accelerator utilization, so the same energy serves more work. And quantizing a model to a smaller numeric format moves less data per operation — directly attacking the data-movement tax — though it must be guarded by evals, because numerics shift.

None of these require new hardware. They require knowing your numbers, which most teams don’t.

The over-retrieval trap#

The single most common waste we see in production AI implementation is over-retrieval. A retrieval-augmented system pulls the top twenty chunks “to be safe,” concatenates them into the prompt, and ships ten thousand tokens of context to answer a question the model could have answered from two hundred. Every one of those extra tokens is energy, latency, and money spent on context the model mostly ignores — and worse, padding the window with marginally relevant text often degrades answer quality by burying the signal.

The fix is unglamorous and measurable. Tighten the retriever so it returns fewer, better chunks; rerank before you stuff the window; and set a context budget per request the same way you’d set a memory limit. A retrieval pipeline that returns three precise chunks instead of twenty cuts inference energy and improves accuracy at the same time. That is the shape of almost every real efficiency win — it’s a data-quality improvement that happens to also cut power, not a tradeoff between the two.

The broader principle: efficiency and quality are usually allies, not enemies. Bloat costs energy and confuses the model. Right-sizing the data flowing through inference tends to help both, which is why measurement pays off twice.

Efficiency-minded AI implementation in practice#

The reason teams fly blind on energy is the same reason they fly blind on cost: nobody owns the measurement. Token counts live in API responses, latency lives in traces, and which model served a request lives in infrastructure metadata. Nobody joins them, so “is this efficient?” has no answer.

Make it answerable. Per request, capture tokens in and out, which model served it, latency by stage, cache hit or miss, and cost. Those are events, and events belong in a system built to aggregate them. The same operational engine we use for Data Platforms work — ClickHouse for the analytics, Airflow for the pipelines, dbt for the modeling — turns raw inference telemetry into a live cost-and-token dashboard. Once that table exists, you can see which features are token-hogs, which requests are over-served by the big model, and where your cache is missing. The same pattern that drives Operational Automation behind a Hospital Management System or a School ERP applies directly: instrument the events, warehouse them, and let the numbers point at the waste.

With that table in hand, efficiency stops being a vibe and becomes a backlog. You sort features by tokens-per-request and attack the top of the list. You A/B a small model against the big one on the easy path and read the quality delta off your evals. You measure cache hit rate and chase it upward. Each is a measurable win in energy, latency, and cost simultaneously — the three that always move together.

The realistic read on the 1,000x claim#

Should you ignore the hardware bets? No — track them. If oscillator computing or the next-generation accelerators deliver even a fraction of their promise, the economics of inference shift, and the teams positioned to take the new option are the ones whose stacks can move. That’s the abstraction-layer discipline: keep the model a portable artifact and the accelerator a swappable backend, so a credible new chip is a benchmark you run, not a rewrite you dread.

But the honest engineering posture is this: a 1,000x simulator result is a promissory note, and your power bill is due now. The biggest near-term efficiency gains for almost every team aren’t in waiting for a new computer — they’re in the tokens you’re sending, the model you’re calling, and the cache you’re not using. Rao’s diagnosis is correct that energy is the ceiling. The response that pays this year is treating inference as a measured, optimizable workload, not a black box you feed and hope.

So the call is plain. Instrument every request. Sort your features by tokens. Route on capability, not habit. Cache like it’s free energy, because it is. Guard quantization with evals. Do that, and you’ll cut your power and cost bill by a real, measured multiple — while the 1,000x chip is still being built.

---

AI efficiency is a data-engineering problem before it’s a hardware one. We instrument inference and turn the waste into a backlog you can burn down. Talk to pdpspectra.