View Events

Event Update From: 08/01/2022 01:36 PM
Title:
Update: MS Exchange Degradation
Summary of Work:
			Update from Microsoft:

"Current status: Our fix to revert the offending update hasn't completed as forecasted, though it's approximately 92 percent complete. Our preliminary testing indicates a corresponding decrease in impacted users, thus most users are no longer impacted by this event. We're continuing to monitor the reversion process as it approaches 100 percent to ensure it completely remediates impact for all users.

Scope of impact: This issue impacts some users attempting to send email messages via the Outlook desktop client and Outlook on the web.

Root cause: A recent update intended to optimize traffic load balancing and improve email transport times introduced a code issue that's leading to impact.

Next update by: Tuesday, August 2, 2022, 2:30 PM (7:30 PM UTC)"

While we do have some reports of this affecting users in our tenant, we haven't seen an increase since Microsoft began to address the problem.
			
Event Start:
07/29/2022 04:51 PM
Event End:


Original Event From: 07/29/2022 04:51 PM
Resolution:
This issue is reported as resolved, and we have not had any new cases reported on campus this week. From Microsoft: "Final status: Our fix to revert the affecting update has completed and we've confirmed via telemetry that message traffic has been restored, confirming impact resolution. Scope of impact: This issue impacted some users attempting to send email messages via the Outlook desktop client and Outlook on the web. Start time: Monday, July 11, 2022, 7:00 PM (7/12/2022, 12:00 AM UTC) End time: Saturday, July 30, 2022, 1:00 PM (6:00 PM UTC) Root cause: A recent update intended to optimize traffic load balancing and improve email transport times introduced a code issue that lead to impact. Next steps: - We're reviewing our update validation and testing procedures to better understand how the code issue was missed to better prevent similar future impact."
Title:
Microsoft Exchange Degradation
Summary of Work:
		
		
Impacted Audience:
Any users of UIUC O365 Tenant
Impact of Work:
		Per Microsoft: "Some users intermittently can't send or open email messages via the Outlook desktop client and Outlook on the web

More info: Affected users receive a Non-Delivery Report (NDR) when attempting to send email messages, but the message still arrives for the intended recipient(s). Depending on the client leveraged, NDRs may contain the following messages:

"This message could not be sent. Try sending the message again later, or contact your network administrator."

"Your message did not reach some or all of the intended recipients."

Users can validate that they're affected by this event by reviewing the NDR and finding "HResult ‏(0x8004010F)" within the error message."

This has been identified as affecting some users in our tenant. Microsoft expects this to be resolved by Monday after reverting a problematic update.
		
Event Start:
07/29/2022 04:51 PM
Event End:
Affected Systems/Services:
Exchange
Affected Campus:
Urbana-Champaign