Infosoft - Checkout reporting 500 Internal Server Error – Incident details

Checkout reporting 500 Internal Server Error

ResolvedMajor outage1 hour 15 minutes

Affected

INFO-Subscription

Major outage from 11:29 AM to 12:16 PM, Partial outage from 12:16 PM to 12:45 PM

Self Service

Major outage from 11:29 AM to 12:16 PM, Partial outage from 12:16 PM to 12:45 PM

  • ResolvedUTC

    This incident has been resolved. We are continuing to monitor the setup and we will start investigating how to prevent the issue in the future.

  • MonitoringUTC

    We have identified the source of the traffic and taken steps to isolate the issue so it does not affect all tenants. We are continuing to monitor the issue.

  • IdentifiedUTC

    We have identified a 80 fold increase in traffic to our checkout applications, which in turn is causing the upstream authentication service to throttle requests.

    We are still investigating how to mitigate the issue.

  • InvestigatingUTC

    We are currently investigating an issue where the Checkout Application is reporting intermediate 500 Internal Server errors.

    Not all tenants are affected at the current time, but the issue seems to be related to the authentication environment.