Infosoft - Authentication Problem – Incident details

All systems operational

Authentication Problem

Resolved
Operational
Started 7 months agoLasted about 22 hours

Affected

INFO-Subscription

Major outage from 2:53 PM to 3:14 PM, Operational from 3:14 PM to 4:41 PM, Degraded performance from 4:41 PM to 12:30 PM

Merchant

Major outage from 2:53 PM to 3:14 PM, Operational from 3:14 PM to 4:41 PM, Degraded performance from 4:41 PM to 12:30 PM

API

Major outage from 2:53 PM to 3:14 PM, Operational from 3:14 PM to 4:41 PM, Degraded performance from 4:41 PM to 12:30 PM

Self Service

Major outage from 2:53 PM to 3:14 PM, Operational from 3:14 PM to 4:41 PM, Degraded performance from 4:41 PM to 12:30 PM

Updates
  • Resolved
    Resolved

    During the night the issue has been resolved by Microsoft and everything is operational again.

    Specifically the underlying service was recovered at 00:27 UTC (02:27 CET). Our metrics indicates everything is operational again.

  • Update
    Update

    The Microsoft Azure support team has now shared a status update on the incident. The issue continues to affect the entire platform.

    They are still trying to figure out the underlying cause.

  • Identified
    Identified

    Unfortunately the issue has not been solved entirely, and we are now experiencing timeouts, but the occasional token request makes it through.

    We have informed Microsoft about the re-occurrence and they are actively working on asolution.

  • Update
    Update

    Microsoft Azure Support has confirmed an issue in their ADB2C infrastructure in Europe.

    The issue should be fixed now, but we will continue to monitor closely.

  • Monitoring
    Monitoring

    ADB2C seems to now be issuing tokens again and authentication works.

    There is already a drop in the error rate on our metrics.

    We will continue to monitor this, and will provide more information if and when we receive it.

    For now the systems are back again, but we are unsure if they will remain available.

  • Investigating
    Investigating

    We are currently investigating an issue with obtaining authentication across the platform.

    Initial log investigation points to Azure ADB2C infrastructure being unavailable for token generation. Regular authentication works, but the underlying API authorization fails across the board.

    We have yet to get any confirmation from Microsoft about the issue and we are working on the problem.

    Unfortunately, when unable to authenticate no other actions are allowed by the platform.