Microsoft unveiled its next-generation custom artificial intelligence processor, the Microsoft Maia 300, on Monday as part of a multi-year effort to power Azure cloud systems with in-house silicon. The silicon focuses on inference rather than model training. It runs models after training. Engineers tailored the design to handle long conversations, multi-step agents, and complex reasoning in tools like Copilot and Teams. It isn’t scheduled for broad service right away, but Microsoft expects the hardware to handle the bulk of its cloud AI demands starting in 2027 [2].
Inside the Microsoft Maia 300 Architecture
The Maia 300 introduces a modular chiplet design that splits compute, memory controllers, and input-output blocks into separate silicon parts rather than keeping them on a single die. Microsoft built the compute chiplets on TSMC’s advanced N2P process node, although it didn’t share fabrication details for the accompanying memory and network blocks. The compute dies use TSMC silicon. To keep memory footprints small during heavy compute loads, the chip supports low-precision datatypes including an enhanced MXFP4 format. This data format compresses numbers in memory, cutting bandwidth use without forcing models to drop long-context reasoning features during prolonged chat sessions [1].
Keeping data close to execution units shaped much of the internal design behind the new chip. Microsoft paired a software-defined dataflow structure with AI-assisted kernel tuning to streamline data movement between compute units and fabric layers. By guiding how data flows through the chip, the system minimizes stalls when running coding agents in GitHub or serving complex prompts through Microsoft Foundry [3].
Unlike general-purpose graphics chips that balance graphics rendering with machine learning, the Maia 300 operates solely as an inference engine. It doesn’t train models. The design team built it for latency-sensitive tasks where software agents must inspect multiple documents and output structured answers in real time [1].
Rack Density and Power Demands
Datacenter deployment centers on a full rack setup packing 96 accelerators into a shared chassis. A complete rack operates within a 167-kilowatt power envelope, delivering sevenfold the token output achieved by an earlier Maia 200 cabinet. Microsoft says the new design produces 40% more tokens per megawatt than competing GPU solutions, although it didn’t name the specific competitor hardware used in its benchmark tests. That density helps cloud operators serve more concurrent AI interactions without building new power feeds. Each rack functions as a unified compute cluster, allowing staff to scale clusters by adding standard cabinets to datacenter rows without altering server hall layouts [1].
The power numbers place the new hardware in direct competition with top industry racks. According to Reuters, Nvidia puts 72 GPUs into its GB300 cabinet while drawing about 142 kilowatts of electricity. Microsoft’s rack houses 96 chips within its 167-kilowatt envelope. Engineers believe the higher chip count will provide enough extra output to offset the larger power draw during business serving workloads [2].
Rack memory reaches massive capacity to prevent network bottlenecks during distributed execution. Memory speed prevents stalls. Each 96-chip cabinet provides 27.6 TB of aggregate HBM capacity with 1.7 PB/s of aggregate HBM bandwidth. By pooling memory across cabinets, Azure can run large frontier models without slicing weights over distant network hops [1].
Memory Bandwidth and On-Chip Scratchpad Design
At the individual chip level, memory speed serves as the main engine for responsive generative interactions. Each Maia 300 chip carries 8 stacks of HBM4 memory to deliver a peak memory bandwidth of 18.0 TB/s while operating within a 950-watt Thermal Design Power (TDP) envelope. Fast memory matters because modern frontier models spend much of their runtime moving weights in and out of compute blocks rather than doing basic math. By reaching 18.0 TB/s, the chip feeds tokens into compute engines fast enough to support real-time chat features without noticeable lag. Microsoft believes this memory setup gives Azure an edge when serving multi-turn chat sessions and long-context documents together [1].
To ease pressure on high-bandwidth memory, Microsoft placed static random-access memory directly on the silicon die. Software uses this on-chip SRAM as a “software-managed scratchpad” to store active token sequences locally. Because reading data from on-die cache draws far less power than querying off-chip memory stacks, the scratchpad helps keep thermal output low during continuous inference workloads [1].
Scaling compute beyond individual chips needs networking that behaves like an integrated part of the dataflow. The chip uses high-speed tensor-parallel links running over standard Ethernet lines. Microsoft says this setup allows smooth scale-up networking from single racks to multi-gigawatt datacenter rollouts [3].
Cloud Rivalry Behind Custom Silicon
Microsoft’s custom chip effort fits into an industry-wide push to cut dependence on Nvidia. Chip costs add up quickly. Google and Amazon already deploy proprietary processors in their server farms, while OpenAI has partnered with Broadcom to develop custom silicon. Nvidia still supplies most of the world’s AI chips, which leaves cloud providers vulnerable to high hardware prices and long delivery queues. By building in-house chips, Microsoft gains firmer control over production schedules and chip costs as business demand for artificial intelligence software grows. The company doesn’t plan to eliminate third-party GPUs entirely, but having its own silicon lets it negotiate better terms with suppliers while securing baseline capacity for its core applications [2].
Optimizing software and silicon together gives Microsoft an efficiency advantage that off-the-shelf chips struggle to match. “It’s basically delivering more intelligence for every watt, every dollar, and every byte of memory by optimizing across the entire system,” said Rani Borkar, who leads hardware systems and Azure infrastructure at Microsoft, when speaking to Reuters. Borkar pointed out that controlling both cloud software and the underlying chip lets engineering teams tune memory access for internal models [2].
Rivalry in cloud chips reaches far beyond American giants, as international tech companies also build rivals to Nvidia. In global markets, suppliers are setting up new fabrication schedules to challenge the market leader, as seen with Huawei advancing its AI chip rollout to capture sovereign datacenter demand [2].
Azure Maia 300 Deployment Timeline for 2027
The Maia 300 builds on foundation work laid by its predecessor, the Azure Maia 200, which already runs production workloads in Microsoft datacenters. That earlier chip handles tasks for Microsoft Copilot, models built by OpenAI, and in-house Microsoft AI models, delivering 1.7 times higher token output per watt than leading graphics processors for built-in AI tools in Teams, Copilot, and Cowork. Microsoft says Maia 200 will also begin serving GitHub Copilot and Microsoft Foundry by the end of this year, proving that in-house hardware can reliably support business software before the newer design arrives. Those live rollouts give hardware teams real-world telemetry that feeds into the design of larger clusters [1].
Scaling supply for the next hardware generation involves deep supply deals that Microsoft has been arranging behind the scenes. In August, The Information reported that Microsoft was in talks with TSMC to produce more than 300,000 Maia 300 chips for deployment in 2027. If rollouts scale into gigawatts as planned, the company could put more than one million Maia 300 chips into server racks during 2027 [2].
Engineers and business customers will get a closer look at the new design soon. San Francisco hosts the event. Microsoft plans to share more system details and deployment schedules for the Maia 300 during its Ignite conference in San Francisco next month. Until then, developers can test early models on current Azure servers as Microsoft readies the racks needed for its 2027 rollout [2].
- ONLINE NEWS Chiappetta, M. (2026, October 12). Microsoft’s Maia 300 AI chip aims to make AI cheaper and faster. HotHardware. [Article Link]
- ONLINE NEWS Constantin, A. M. (2026, October 12). Microsoft says Maia 300 chip will run its AI services from 2027. The Next Web. [Article Link]
- ONLINE NEWS Moore, M. (2026, October 12). Microsoft unveils Maia 300, its most powerful AI chip yet. TechRadar. [Article Link]
