Original URL: http://www.theregister.co.uk/2013/02/15/microsoft_azure_sql_reporting_cleanup/

Microsoft still reviving Azure SQL Reporting after MONDAY FAIL

'Procedural operations error' leads to lengthy cloud cleanup

By Jack Clark

Posted in Cloud, 15th February 2013 22:06 GMT

Four days have passed since a "procedural operations error" downed Azure SQL Reporting in Microsoft's East US data center, and Redmond is still trying to restore customer data.

After saying on Thursday that full restoration from Monday's fail would have occurred by Friday, the recovery date has slipped again, according to updates posted by the company to the Windows Azure Service Dashboard.

No data was harmed in the incident, but getting it back is taking a long time.

"Restoration progress continues," Redmond wrote at 9am PST. "Current estimated recovery time has been extended slightly to 2/16/2013 at 12:00 AM UTC. We will provide updates if this recovery time changes. We apologize for any inconvenience this causes our customers."

It is taking longer than expected to restore everything because "the engineering team needs to take specific and necessary steps to ensure full data recovery on the affected cluster," Microsoft's director of comms for the server and tools business, Cori Hartje told The Reg in an e-mailed response to our queries.

The answers indicate that something may have gone wrong with a bit of data-center hardware, as Hartje notes that: "Microsoft writes multiple replicas of user data to multiple devices ensuring the data is still available even if single or multiple catastrophic hardware failures take place."

Microsoft is not restoring the data from tape, she said.

Microsoft expects to have brought everything back by tomorrow. After that, affected customers can contact Azure support for a root-cause analysis, the company said. ®