April 6, 2022
We’ve experienced an incident on our platform in Amsterdam that caused an outage.
The incident was short but affected many customers. 74,5% of all the BGP-sessions on the platform were interrupted. The incident occurred between 10:20 and 10:45. The incident is now resolved.
The incident was caused by a regular test of a line card in preparation for standard maintenance on the platform. Due to an unfortunate set of circumstances, an action by an AMS-IX operator set the wrong settings on one of the ports, which created a loop on the production network. This loop disrupted the routing of packets across the AMS-IX platform, causing connected customers to experience outages.
The issues were resolved by disabling the loop. Our customers and members were updated on the root cause via our tech-l mailinglist yesterday.
We are reviewing our internal procedures to make sure that we will limit the possibility of a similar event happening in the future. We sincerely apologize for any inconvenience caused by the outage.