On October 12, 2026, Microsoft unveiled the Maia 300, its third-generation custom AI chip — and, in the words of CEO Satya Nadella, the company's "biggest bet yet" on its own silicon. The announcement, made at the OCP Global Summit in San Jose, was not about raw speed. It was about something far less glamorous and far more important to the future of AI: energy efficiency.
Microsoft claims the Maia 300 delivers 40 percent more tokens per megawatt than leading GPU-based systems, and up to seven times the rack-scale token throughput of its predecessor, the Maia 200. If those numbers hold up in real deployments — and that is a meaningful if — the chip could reshape the economics of running AI at scale. Here is what Microsoft announced, what the numbers mean, and why a chip designed to save electricity matters more than it sounds.
Why Microsoft is building its own AI chips
Every major technology company is now in the chip business, and for the same reason: the companies that run the largest AI workloads no longer want to depend entirely on off-the-shelf processors. Designing your own silicon lets you tailor the hardware to exactly the software you run — and at the scale Microsoft operates, even a single-digit efficiency gain translates into hundreds of millions of dollars in saved electricity.
Microsoft's Maia program began with the Maia 100, announced in late 2023 as a 5nm inference accelerator. The Maia 200 followed in January 2026, built on TSMC's 3nm process with 216GB of HBM3e memory, and it already serves Microsoft Copilot workloads in production across two US regions. The Maia 300 is the third step in that lineage — and the first built on 2nm-class technology.
This is part of an industry-wide shift. Foundries like GlobalFoundries are reshaping their process roadmaps to serve exactly this kind of custom-silicon demand, as we explored in our look at GlobalFoundries' FDX Fusion chips — a sign that bespoke AI hardware is becoming the norm rather than the exception. Microsoft's decision to keep the Maia family entirely in-house (third parties cannot rent Maia accelerators, unlike Google's TPUs or AWS Trainium) shows how strategically the company now treats its silicon.
Inside Maia 300: the specs that matter
The Maia 300 is not a single monolithic chip. It is a multi-chiplet design: four compute chiplets built on TSMC's N2P process (a 2-nanometer-class node), eight stacks of HBM4 high-bandwidth memory, plus separate chiplets for device control and data input/output. The compute, memory-controller, and I/O functions are physically separated — an approach that lets Microsoft mix and match the newest process technology for the parts that need it most.
Memory first: 288GB of HBM4 at 18 TB/s
The headline number is memory, not compute. Each Maia 300 carries 288GB of HBM4 — up from 216GB of HBM3e on the Maia 200 — with a peak memory bandwidth of 18 TB/s. That is roughly 2.5 times the bandwidth of the previous generation, inside a 950-watt power envelope (up from 750W on the Maia 200).
Why does this matter more than raw compute? Because modern AI inference is mostly memory-bound. When a model generates a long conversation or works through a multi-step task, it spends most of its effort reading the model's weights and the conversation's context cache from memory, over and over. Doubling the memory bandwidth per watt — which is effectively what Maia 300 does — is where the cost of serving each token actually comes down.
Rack-scale design: 96 chips, one superpod
Microsoft is not really selling a chip; it is selling a system. A full Maia 300 rack packs 96 accelerators into a 167-kilowatt power envelope, delivering 27.6TB of total HBM capacity and 1.7 PB/s of aggregate memory bandwidth across a high-speed scale-up domain. Racks are grouped into superpods of up to 12,288 accelerators connected over Ethernet.
The rack design matters because AI inference increasingly happens at enormous scale, spread across many accelerators working in concert. The up-to-7x rack-scale throughput claim versus the Maia 200 comes from this combination of faster chips, twice as many accelerators per rack (96 versus 48), and faster interconnects — not just from the silicon alone.
Tokens per megawatt: the metric that matters
Of all the numbers Microsoft published, "40% more tokens per megawatt" is the one worth understanding, because it reflects how data-center economics actually work. A token is the basic unit of AI output — roughly three-quarters of a word. A megawatt is a million watts of electricity. Tokens per megawatt answers the only question a data-center operator really cares about: how much AI work do I get for each unit of power I buy?
This framing is deliberate. Data-center power is becoming the scarcest resource in the AI boom. Utilities are being asked to deliver hundreds of megawatts to single campuses; some AI data centers are already being built with their own dedicated power plants. In that world, a chip that is 40% more efficient per watt does not just save money — it lets Microsoft deploy more AI inside the same power budget, which is the binding constraint on growth.
Production deployments are planned to begin in 2027, scaling to what Nadella called "multi-gigawatt deployments" — enough capacity that Maia will become, in his words, "the primary inference system for our own products and services." Reuters has reported that Microsoft may deploy more than one million Maia chips in 2027, underscoring the scale of the bet.
"Today we are announcing our biggest bet yet on our own silicon. Starting next calendar year, we will scale our Maia family to multi-gigawatt deployments, making it the primary inference system for our own product and services." — Satya Nadella, October 12, 2026
Reading the fine print: claims versus reality
A careful reader should treat every number in this announcement the way Microsoft itself framed them: as vendor claims, not independent results. As of mid-October 2026, no independent benchmarks, pricing, or hands-on tests of the Maia 300 exist. The chip will not enter production deployment until 2027.
There are two specific caveats worth knowing. First, Microsoft has not identified the "leading GPU-based systems" it compares against for the 40% tokens-per-megawatt figure. Without a named baseline, the claim is directionally interesting but impossible to verify or compare against rivals — including NVIDIA's Rubin platform, which pairs its own accelerators with the same 288GB of HBM4. Second, the 7x rack-scale throughput figure compares a full Maia 300 rack against a Maia 200 rack, mixing chip improvements with denser packaging (twice the accelerators per rack), so it is not a measure of single-chip speed.
None of this makes the claims false — only unverified. The underlying architecture choices (memory-heavy, inference-first, rack-scale co-design) are sound responses to real bottlenecks, and independent analysts who have studied the technical disclosure broadly agree that inference workloads are exactly where a custom chip pays off.
Why inference is the new battleground
For years, the AI hardware story was about training — the massively expensive process of teaching a model. But training a model happens once; running it (inference) happens billions of times a day, forever. As AI shifts from research labs into everyday products — Copilot in Office, conversational agents, coding assistants, the ambient AI in devices like Amazon's Bee AI wearables and devices — inference has become the dominant cost, and the dominant energy drain.
That is why the Maia 300 is inference-first by design. Microsoft says it is aimed at "long-context reasoning, latency-sensitive conversational experiences, and sustained, multi-step agentic tasks" — the workloads that define the current wave of AI products, where a single user request can trigger minutes of continuous token generation. These are precisely the workloads where memory bandwidth, not raw compute, determines both latency and cost.
The chip also reflects a maturing design philosophy. Compute per watt barely moves between generations (roughly 8% better in FP4 teraops per watt), while memory bandwidth per watt doubles. Microsoft is betting that the future of AI efficiency is won in the memory system — by keeping data close to compute and moving it as little as possible — rather than by piling on more arithmetic units.
What this means for the AI cost race
The Maia 300 announcement is best read as a statement of intent. Microsoft is telling the market that it plans to own its AI economics end to end — from the silicon to the software to the data center — rather than renting that stack from someone else. If the efficiency claims materialize, Microsoft can serve more Copilot queries per dollar of electricity than any competitor buying commodity hardware.
For the rest of us, the practical takeaway is simpler: the cost of AI is increasingly a hardware story, not a software one. The models get the headlines, but the real competition — the one that decides which AI products can be offered cheaply, at scale, without blacking out the power grid — is being fought in chiplet layouts, memory stacks, and tokens per megawatt. The Maia 300 is Microsoft's biggest chip yet. Whether it is also its best is a question that will be answered in 2027, one rack at a time.