Azure TITSUP caused by INFINITE LOOP
Fat fingered geo-block kept Aussies in the dark
A global balls-up of Redmond's Azure's caused by an infinite loop bug might have crept by those dwelling in the antipodes thanks to a seemingly fat-fingered admin who geo-blocked the region from reading the news.
The TITSUP (Total Inability To Support Usual Service) on the Azure cloud service Azure, starting at about 11 am GMT yesterday, may have sent some remote sysadmins scrambling for an answer longer than their British, European and American counterparts: Microsoft's statement was for some time inaccessible by users in Australia, New Zealand and many parts of Asia.
Azure bod Jason Zander apologised to everyone (outside of the antipodes) for a necessary server restart in a blog that has been unlocked for all, following a Vulture South inquiry.
"As part of a performance update to Azure Storage, an issue was discovered that resulted in reduced capacity across services utilizing Azure Storage, including Virtual Machines, Visual Studio Online, Websites, Search and other Microsoft services," Zander told the Northern Hemisphere.
"During the roll out we discovered an issue that resulted in storage blob front ends going into an infinite loop, which had gone undetected during fighting (testing).
"The net result was an inability for the front ends to take on further traffic, which in turn caused other services built on top to experience issues."
The Redmond restart has scrubbed the problem out of existence and restored services for all but an undefined small few.
"When we have an incident like this, our main focus is rapid time to recovery for our customers, but we also work to closely examine what went wrong and ensure it never happens again," Zander said.
Microsoft has not yet responded for a request for comment regarding the cause of the geo-block, but lifted it sometime after Vulture South asked why it was there. ®