Google has now said that lightning strikes caused a loss of power to some of its storage systems, resulting in the persistent disk errors. “However, in a very few cases, recent writes were unrecoverable, leading to permanent data loss on the Persistent Disk”.
Advertisement
The data center powers the Google Compute Engine, a service for business customers who rely on Google’s massive servers to perform high-powered computing tasks. A tiny fraction of the persistent-disk space in the zone lost some data permanently: 0.000001 percent, according to Google. There was no effect on Standard Persistent Disks created after 09:19. However, it stressed to customers that “GCE instances and Persistent Disks within a zone exist in a single Google data center and are therefore unavoidably vulnerable to data center-scale disasters”. “We have conducted a thorough analysis of the issue, in which we identified several contributory factors across the full range of our hardware and software technology stack, and we are working to improve these to maximise the reliability of GCE’s [Google Compute Engine] whole storage layer”.
“This outage is wholly Google’s responsibility”, said the search giant. “The durability of storage is our highest priority”, the company states.
Fans of Douglas Adams” Hitchhiker’s Guide series will know that “Belgium’ is the most unspeakably rude word in the galaxy, customers of GCE now understand why. It is unknown which clients have been affected by the outage and have now got to deal with the data loss. And when lightning hits a building multiple times, it doesn’t need to hit in exactly the same spot each time to cause extra damage.
Justin Gale, project manager for the lightning protection service Orion, said lightning could strike power or telecommunications cables connected to a building at a distance and still cause disruptions.
“The cabling alone can be struck anything up to a kilometre away, bring [the shock] back to the data centre and fuse everything that’s in it”, he said.
Advertisement
And James Wilman, engineering sales director for the data centre consultants Future-Tech, said that though such data centers are designed to withstand lightning strikes via a network of conductive lightning rods, it was not impossible for strikes to get through.
