On Monday, Google’s cloud services went offline, leaving customers unable to access services for 18 minutes.
Google has now announced that it will refund its affected customers in credits for the problem, after investigations confirmed that the issue was caused by Google engineers performing a network update to their systems.
In the recent company statement, Benjamin Treynor Sloss, Vice President of Engineering for the corporation, stated that the issue was not at risk of being repeated, explaining that:
“[I]nbound internet traffic to Compute Engine instances was not routed correctly, resulting in dropped connections and an inability to reconnect.
The loss of inbound traffic caused services depending on this network path to fail as well, including VPNs and L3 network load balancers. Additionally, the Cloud VPN offering in the asia-east1 region experienced the same traffic loss starting at an earlier time of 18:14 Pacific Time but the same end time of 19:27.”
The issue didn’t affect the Google App engine, Cloud storage and other Cloud platform products. The statement continued:
“It is our intent to enumerate all the lessons we can learn from this event, and then to implement all of the changes which appear useful. As of the time of this writing in the evening of 12 April, there are already 14 distinct engineering changes planned spanning prevention, detection and mitigation, and that number will increase as our engineering teams review the incident with other senior engineers across Google in the coming week.”