Microsoft has put a unit on the next stage of its AI build-out, and the unit is the gigawatt. In a blog post on 12 October, Scott Guthrie, the company's executive vice president for cloud and AI, introduced Azure Maia 300, the third generation of Microsoft's own AI accelerator, and wrote that the Maia family will be scaled to multi-gigawatt deployments from 2027. Maia is to become the main inference system for Microsoft's products and services, the hardware that answers requests once a model has been trained.
The Rack Is the Unit
The figures are Microsoft's own and nobody outside the company has measured them yet. Maia 300 is built from several chiplets - compute on TSMC's N2P process, memory controllers and I/O on separate dies - with eight stacks of HBM4 memory giving 18 TB/s of peak bandwidth inside a 950 W thermal design power per accelerator. Ninety-six of them go into one rack within a 167 kW power envelope, holding 27.6 TB of high-bandwidth memory. Microsoft claims up to 40 per cent more tokens per megawatt than leading GPU-based systems and up to seven times the rack-level throughput of Maia 200. Production deployments are planned for 2027, and more is promised at the Ignite conference in San Francisco next month.
Maia 200, launched in January, is already in production serving Microsoft Copilot and the company's own model development, with GitHub Copilot and Microsoft Foundry to follow by the end of the year. Unite.AI's report recalls that it went in first at the US Central region near Des Moines, Iowa, with US West 3 near Phoenix next; Microsoft says it is now deploying in the United States, Europe and Australia.
Tokens per Megawatt
The telling number for the power side is the yardstick Microsoft chose: tokens per megawatt. Efficiency is stated against the electricity supply rather than the chip, which says where the constraint sits. A rack of 167 kW is a small industrial load on its own, a hall of them is a substation, and multi-gigawatt is the language of utilities rather than of IT departments. Converge Digest's report points to the cooling and power delivery around the rack as the other half of the design at this density.
What Stands Behind an Inference Gigawatt
A service that answers Copilot requests is not a load anyone lets drop when the grid does, so the standby generation that goes with data centres of this class goes with these too. Nothing in the post says where the gigawatts will be built or how the sites will be fed, and backup power is not mentioned at all; what it does say is that the load arrives from 2027 and runs around the clock. For the people who package and permit generator yards the questions are the familiar ones at a larger scale: how many sets a campus needs for its tier, how a permit treats the test runs of a fleet that size, and how the yard is kept exercised under load without touching the live IT load. This week's DOE order to PJM is a reminder that the grid may ask that standby yard for help as well, inference load and all.
Sources: Microsoft (Official Microsoft Blog), Converge Digest, Unite.AI