Microsoft Corp. unveiled Maia 300, a new internally developed accelerator tuned for artificial intelligence inference, the company said. Microsoft estimates the chip can process 40% more tokens per megawatt than leading GPU-based systems, a category that likely points to products from Nvidia Corp. The company said the Maia line will eventually serve as its main source of inference capacity.
Maia 300 is built from four computing chiplets manufactured on Taiwan Semiconductor Manufacturing Co.'s newest two-nanometer node. Eight HBM4 modules surround the chiplets and hold 288 gigabytes combined, matching the capacity of Nvidia Corp.'s flagship Rubin accelerator. Maia 300 trails Rubin by roughly 20% in memory bandwidth, which measures how much fast data can move to and from RAM.
Inference chips typically shuttle data repeatedly between computing and memory circuits, and that traffic slows processing. Microsoft said Maia 300 uses a dataflow architecture meant to cut down on data movement, with computing chiplets holding information in an internal SRAM cache whenever possible instead of sending it out to the HBM stacks. When HBM use is unavoidable, the chiplets route data to a nearby memory stack rather than one at the far end of the chip, which is faster.
Microsoft executives Rani Borkar and Saurabh Dighe wrote in a blog post that narrower representations reduce memory footprint and data movement, letting the system handle larger models, longer contexts and more concurrent sessions inside the same power envelope. Maia 300 also compresses data into lightweight formats such as MXFP4 to speed movement inside the chip. A device control chiplet manages power delivery and telemetry for the computing and memory modules, while two auxiliary chiplets handle data entering and leaving the accelerator.
Microsoft will deploy Maia 300 in trays holding four accelerators each, placed in liquid-cooled racks totaling 96 accelerators. The company expects those racks to deliver about seven times the token throughput of a system built on its previous-generation Maia 200 chip, which debuted in January and is already in production. Microsoft plans to group the racks into superpods of 12,288 accelerators linked by a custom Ethernet-based network stack that can shift bandwidth between in-rack traffic and inter-rack communications.
Typically, chips exchange data differently inside and across racks, forcing a switch in networking modes that relies on the host server's central processing unit or a separate network chip. Maia 300 instead uses the same communication semantics for both, removing that hardware and making clusters more efficient. Borkar and Dighe said jobs can draw on capacity across rack boundaries, reducing stranded accelerators when workload sizes do not line up with the physical rack layout. Microsoft plans to begin production deployment of Maia 300 next year.
More AI news from TechManNews.
