Forty per cent more tokens, against an unnamed GPU

Microsoft on Monday unveiled Maia 300, the third generation of its in-house AI accelerator, and said it will scale the Azure Maia family to multi-gigawatt deployments, Scott Guthrie, the company's executive vice president for cloud and AI, wrote in a blog post. "We expect Maia 300 to deliver 40% more tokens per megawatt compared to leading GPU-based systems," Guthrie wrote. The post does not name the systems, models or workloads behind that comparison, and Microsoft presents the figure as an expectation, not a measured result.

"Maia 300 will begin production deployments in 2027," the post says. Guthrie wrote that Microsoft will "scale our Azure Maia family to multi-gigawatt deployments starting in 2027, making Maia the primary inference system for Microsoft products and services". Computerworld reported that Microsoft plans to deploy the chip "after the turn of the year"; Microsoft's own post gives only the year. Computerworld also said, citing The Verge, that the chip will make Microsoft less dependent on Nvidia and AMD. Microsoft's post frames the effort around efficiency and capacity and does not name suppliers.

Rani Borkar and Saurabh Dighe of Microsoft's Azure hardware group set out the design in a separate technical post. Compute chiplets are built on TSMC's N2P process, and each accelerator has eight HBM4 memory stacks providing 288 GB of capacity and 18 TB per second of bandwidth, within a 950-watt power rating given in Guthrie's post. A rack holds 96 accelerators in a 167-kilowatt envelope, and a superpod can span up to 12,288 accelerators across up to 128 racks. By our arithmetic, 96 accelerators at 288 GB each gives 27.6 TB of memory per rack, which matches the rack figure Microsoft gives.

Most of Microsoft's comparisons are with its own previous chip. The technical post says a full rack delivers up to seven times the token throughput of Maia 200 and gives performance per watt as "more than 2.5 times" the earlier chip's in one passage and "up to 2.5x" in another. It gives no tokens-per-megawatt figure, so the 40 per cent comparison with GPU systems rests on Guthrie's post. Guthrie also wrote that Maia 200 delivers up to 1.7 times more tokens per watt for AI features in Teams, Copilot and Cowork than "leading GPU infrastructure", again without naming it. Neither post publishes a benchmark method.

Maia 200, the previous generation, launched in January, Computerworld reported. Guthrie's post says it now serves Microsoft production workloads including Copilot and supports the company's MAI model team, with GitHub Copilot and Microsoft Foundry to follow by the end of the year. It runs OpenAI models, Microsoft's MAI models and open-source model families, the post says, and is deploying across the United States, Europe and Australia.

The announcement fits a pattern in which large cloud customers of the GPU makers design their own silicon for inference, the workload that grows with every user query. As we reported, the largest buyers of AI accelerators now ship their own chips at volume, which changes supplier leverage more than it changes raw performance. Microsoft's plan to make Maia the primary inference system is about its own products and services. Neither post says whether Azure will change its purchases of third-party accelerators for customers.

The posts also leave open what Maia 300 costs to build and run. The technical post says the chip is designed to maximise "useful tokens per watt and useful tokens per dollar", but Microsoft publishes no price or cost comparison. Custom silicon is also a supply-chain commitment, as our coverage of Broadcom's forecast showed. Microsoft said it will share more at its Ignite conference in San Francisco next month.