Skip to content
Komposite News
Markets · Technology · Capital
Thursday, September 10, 2026Global EditionGot a tip?
Outages

Microsoft traces its mail outage to a July fix

A Windows change Microsoft began deploying in July sat dormant for more than three months before taking down the gateway in front of Exchange Online, its preliminary report says. More than 80 per cent of the gateway's servers stopped responding.

Komposite News Staff
November 17, 20264 min read
A rack-mounted network switch with its ports filled, surrounded by a dense tangle of blue and grey network cablesJ.smith, CC BY-SA 2.5, via Wikimedia Commons
A rack-mounted network switch surrounded by a tangle of blue and grey network cables, in a file photograph from 2007.Photograph via Wikimedia Commons

A Windows fix that Microsoft began deploying in July caused the Exchange Online outage of 12 and 13 November, the company says in a preliminary post-incident report dated 15 November.

A surge of Teams traffic and a block of servers already out of service pushed the gateway in front of Exchange past breaking point, the report says. More than 80 per cent of the gateway's servers stopped responding at the worst point, and mail access was disrupted for most users for 23 hours and 50 minutes.

The seven-page report, a copy of which was published by the Microsoft 365 site 365Fanatic, describes "a latent defect" that made Windows servers "overwhelmed and unresponsive under certain load conditions" through "lock contention inside an internal Windows process". The fix was "prepared specifically for Microsoft 365 networking gateway servers", and Microsoft says "neither the underlying defect nor the fix applies to broader Windows deployments". The report gives the outage as 16:20 UTC on Thursday to 16:10 UTC on Friday, 23 hours and 50 minutes by our arithmetic.

Why did a July change fail in November?

Because the load changed. The report says the change "first started deployment in July of this year" and that "For more than three months, the change was deployed worldwide without impact", between 104 and 134 days by our count depending on the July start. Load testing missed it: "The contention issue is worsened on machines with more cores and newer hardware; whereas our load testing tended to focus on weaker hardware with fewer cores."

Two things then lined up. About 15 hours before the outage, at 01:07 UTC, Teams failed over from Microsoft's alternative gateway onto the "primary" one that also carries Exchange Online, which the report calls "part of normal operations". And "nearly 14% of server capacity for the networking gateway was out of service due to an unrelated issue from the previous day". The servers began failing "during the peak window of traffic in the United States". When failing servers handed their traffic to healthy ones, those failed too: "In less than two hours, the previously unknown defect was encountered across our fleet", and "At its lowest, more than 80% of servers were unresponsive."

Times from Microsoft's preliminary post-incident report, dated 15 November 2026; elapsed hours are Komposite's arithmetic.
Time (UTC)What Microsoft's preliminary report says happenedHours after the 16:20 alert
01:07 Thu 12 NovAutomated failover moves Teams traffic onto the gateway servers shared with Exchange Online−15.2
16:03Anomaly monitor sees connectivity problems rising−0.3
16:20High-priority alert; incident EX1550543 posted0
19:00All deployments and configuration changes paused2.7
00:30 Fri 13 NovConnection negotiation identified as the main cause of access failures8.2
07:20Update reverting the defective change applied to part of the capacity15.0
14:17Global webmail availability back to expected levels22.0
16:10No active customer impact; extended monitoring begins23.8

The gateway was built to contain this kind of failure, and that did not work either. Microsoft says the gateway is made of "thousands of 'capacity units'" grouped into regions, but "due to a separate issue, the normal bulkheads between these different partitions did not stop the cascading failure", which is how the fault spread worldwide. The timeline shows the first night's fixes failing: at 22:00 UTC on Thursday Microsoft's telemetry "confirmed that the actions we'd taken to restore service had not been effective", and at 23:55 it found that servers returned to service "initially remained healthy but then become unresponsive again upon receiving traffic". The report does not describe the separate issue that weakened the bulkheads.

Teams, OneDrive and SharePoint moved off the faulty gateway, which is why they recovered first, but "not all services, like Exchange Online, can fail out of this network gateway for scale reasons". Restarted servers were "immediately overwhelmed by waiting traffic", and because the defect was months old, "our normal rollback, flighting, and feature flags were unable to quickly revert the change". Engineers took eight hours and ten minutes from the first alert to identify connection negotiation as the problem, and 15 hours to begin applying an update that put the code back to its earlier state, by our arithmetic from the report's timeline.

Which start time is right?

The report says 16:20 UTC, but Microsoft's own incident record still disagrees. The closed record for EX1550543, as copied by the University of Pennsylvania on Tuesday, carries a Start Time field of 10:05 UTC, the time Komposite reported on Saturday as a six-hour revision. Its final status text and the report both say 16:20. The record's status changed to "postIncidentReviewPublished" with a note at 04:47 UTC on Monday that "A post-incident report has been published".

Komposite's comparison of Microsoft's service-health record as copied by the University of Pennsylvania with the preliminary report. Uptime is our arithmetic on 43,200 minutes; all three fall in the same credit tier of Microsoft's service-level agreement (below 99 per cent, above 95).
Where the start time appearsStart (UTC, Thu 12 Nov)Outage to 16:10 FriUptime for a user out throughout, 30-day month
Incident record as first logged (our 13 November reading)16:1723h53m96.68%
Start Time field of the closed record, still shown on 17 November10:0530h05m95.82%
Final status text and preliminary report16:2023h50m96.69%

For customers' service credits the difference does not change the tier, by our arithmetic, though it changes the minutes counted, which Microsoft measures per user across a tenant. As we reported on Friday, customers have to claim those credits from Microsoft; they are not paid automatically.

The report lists five repair items as completed, including guardrails "for traffic distribution to avoid cascading impact" and more monitoring, and says a "comprehensive list of repair items will be delivered as part of the final PIR" on 20 November, the date The Register also gave. 365Fanatic's summary says the report runs to seven pages. The final report is the next fixed date, five business days after Friday's end time.

MORE ON THIS STORY