Google Cloud’s load balancers suffered a connectivity problem for 18 hours, affecting virtual machines across the US, Europe and Asia.
The incident was first reported yesterday at 00:52, and was unresolved by 19:18.
Google wrote in a statement at 06:00: “We are determined the infrastructure component is responsible for the issue and mitigation work is currently underway.”
At 08:30 the message changed: “We have identified the event that triggers this issue and are rolling back a configuration change to mitigate this issue.”
Google stated it was carrying “further measures to completely resolve the issue”, once the change was implemented, half an hour after.
Afterwards, Google advised users to do the following:
- Create a new TargetPool.
- Add the affected VMs in a region to the new TargetPool.
- Wait for the VMs to start working in their existing load balancer configuration.
- Delete the new TargetPool.
- DO NOT delete the existing load balancer configure, including the old target pool.
- Create a new ForwardingRule.
According to The Register, Google admitted that it was source of the issue through an update of its own.
Google also broke its own cloud in April 2016, July 2016, August 2016 and September 2016.
Written by Leah Alger
