Office 365 service degradation for users of Firefox
Incident
Report for Ohio University
Resolved
Per Microsoft, the Office 365 vs. Firefox issue was marked resolved Wednesday, May 31, 2017 at 11:00 PM EDT.
Preliminary root cause: An update containing a renewed security certificate related to Online Certificate Status Protocol (OCSP) did not propagate across the environment as intended, resulting in old certificate data being cached. This led to Secure Sockets Layer (SSL) connection errors when using the Mozilla Firefox browser due to the unique way it handles OCSP stapling.
Next steps:
- We're reviewing our monitoring services to find ways to reduce detection time and to improve our automated recovery processes.
- We're reviewing our update procedures to help catch this kind of problem during our testing cycle.
::
If you encounter difficulties seemingly related to this incident, please advise the IT Service Desk at 740.593.1222.
Preliminary root cause: An update containing a renewed security certificate related to Online Certificate Status Protocol (OCSP) did not propagate across the environment as intended, resulting in old certificate data being cached. This led to Secure Sockets Layer (SSL) connection errors when using the Mozilla Firefox browser due to the unique way it handles OCSP stapling.
Next steps:
- We're reviewing our monitoring services to find ways to reduce detection time and to improve our automated recovery processes.
- We're reviewing our update procedures to help catch this kind of problem during our testing cycle.
::
If you encounter difficulties seemingly related to this incident, please advise the IT Service Desk at 740.593.1222.
Update
Update from Microsoft on the Firefox access issue reported yesterday:
Current status: We're now over 98 percent complete with our remediation actions and the vast majority of users should no longer be experiencing impact. We'll continue restarting the remaining systems throughout the day and we are confident that all systems will have the updated certificate installed by 11:00 PM EDT on May 31, 2017. At that time, we'll perform final testing and validation to confirm full remediation.
Current status: We're now over 98 percent complete with our remediation actions and the vast majority of users should no longer be experiencing impact. We'll continue restarting the remaining systems throughout the day and we are confident that all systems will have the updated certificate installed by 11:00 PM EDT on May 31, 2017. At that time, we'll perform final testing and validation to confirm full remediation.
Monitoring
Office 365 service degradation for users of Firefox
Services impacted: Exchange Online (Catmail); OneDrive for Business, including Office Web Apps
User Impact: Users may be intermittently unable to connect to various Office 365 products and services using the Firefox browser.
More info: Affected users may see the following error when using Mozilla Firefox to connect to Office 365 services: "`Invalid OCSP signing certificate in OCSP response.`" Browsers other than Firefox are not affected by this issue.
While we're focused on resolving the issue, users may connect to the affected sites via other browsers, such as Chrome, Edge, Safari, or Internet Explorer. We're restarting services in an attempt to clear the stale data from the affected environment. We are limited in how fast we can perform this work due to the risk of causing more severe impact. While our remediation actions are in progress, we strongly encourage affected customers to use a different browser if they are able to do so. Users will continue to experience service restoration as affected systems are restarted.
Preliminary root cause: We've determined that stale Online Certification Status Protocol (OCSP) certificate data was preventing access to the services or features.
Scope of impact: Customer reports indicate that many users will likely experience impact related to this event. Our analysis indicates that this affects any users connecting to Office 365 web services via the Firefox browser.
Services impacted: Exchange Online (Catmail); OneDrive for Business, including Office Web Apps
User Impact: Users may be intermittently unable to connect to various Office 365 products and services using the Firefox browser.
More info: Affected users may see the following error when using Mozilla Firefox to connect to Office 365 services: "`Invalid OCSP signing certificate in OCSP response.`" Browsers other than Firefox are not affected by this issue.
While we're focused on resolving the issue, users may connect to the affected sites via other browsers, such as Chrome, Edge, Safari, or Internet Explorer. We're restarting services in an attempt to clear the stale data from the affected environment. We are limited in how fast we can perform this work due to the risk of causing more severe impact. While our remediation actions are in progress, we strongly encourage affected customers to use a different browser if they are able to do so. Users will continue to experience service restoration as affected systems are restarted.
Preliminary root cause: We've determined that stale Online Certification Status Protocol (OCSP) certificate data was preventing access to the services or features.
Scope of impact: Customer reports indicate that many users will likely experience impact related to this event. Our analysis indicates that this affects any users connecting to Office 365 web services via the Firefox browser.
Ohio University Contact Information:
Ohio University | 1 Ohio University | Athens OH 45701 | 740.593.1000
ADA Compliance
| © 2019
Ohio University. All rights reserved.
