Original URL: http://www.theregister.co.uk/2011/01/07/microsoft_explains_hotmail_outage/

Microsoft confesses to New Year Hotmail blunders

Claims nothing deleted, admits emails may have 'bounced'

By Kelly Fiveash

Posted in Software, 7th January 2011 12:28 GMT

Microsoft's Hotmail saw the New Year in with a whimper, after thousands of its users were unable to access their web-based email accounts.

Yesterday the company explained what went wrong, offered a relatively minor apology given the scale of the cockup, and said it would endeavor to prevent similar mistakes in the future.

Worse still, it also coughed to separate "data loss issues" on the software vendor's increasingly tarnished Hotmail service.

Microsoft's Mike Schackwitz blamed a script error on the first outage that affected 17,355 users, who had reported a major problem that downed some mailboxes on 30 December.

However, Redmond didn't take full action until two days later when it finally raised the priority of the issue, after reports continued to flood in from unhappy users.

Microsoft said the script error removed the directory records "of a small number of real user accounts along with a set of test accounts".

It claimed that no email messages or folders had been deleted when the cockup occurred, but confessed that "the inbox location in the directory servers was removed".

Thousands of users hit by the error were greeted with a new mailbox and a "Welcome to Hotmail" message when they tried to log into their accounts.

Microsoft wonks spent the past few days restoring the inboxes for the users affected, and it finally reunited everyone with their email on 5 January - that's a mighty six days after Hotmailers raised the alarm.

Despite the slow response, Schackwitz said he was "happy to report that no user data was permanently lost in this particular incident".

But here's the rub:

"The only unfortunate exception to this statement is that, if you were affected by this incident and you didn't sign in to your account between the time of the incident and the time your account was restored, then any messages sent to your account during that time would have bounced."

Microsoft is now rejigging how it provisions and removes test accounts in Hotmail by using a separate code path, but many will wonder why such a system wasn't in place already.

It's also changing the firm's "issue alert process" in Hotmail so missing data issues will be responded to more quickly. But again, critics will ask why this wasn't standard procedure for such reports. ®