O2's titsup network struggles to find its feet
Have you tried switching it off and on again?
Regcast training : Hyper-V 3.0, VM high availability and disaster recovery
O2's mobile network is struggling to its feet after a 19-hour outage left thousands of customers unable to make or receive calls. This morning 2G should be working again but 3G remains patchy.
That's according to the sluggish status page, which, between apologies, claims the 3G service is slowly being revived.
The telco admits that punters might get a better connection by switching their mobiles off and on, or perhaps just temporarily disabling 3G networking and dropping down to the restored 2G system.
The outage kicked off at lunchtime yesterday, but as it was related to O2's sudden inability to attach handsets and devices to its network, it didn't hit everyone at the same time.
In the normal run of things phones periodically reattach to the network, and as that process failed customers were gradually booted off while other users remained connected, much to their irritation.
The outage also hit the virtual networks camped on O2's infrastructure, including Tesco Mobile and O2-owned GiffGaff.
Some El Reg readers are reporting successful connections, so the situation is gradually improving. The telco told us in a statement:
Customers affected may wish to try switching their mobile phones off and on as service returns. We are sorry again for the inconvenience this has caused and can provide reassurance that we continue to deploy all possible resources and will do so until full service is restored.
Once the network is up we'll be asking O2 what went wrong and (more importantly) what's being done to ensure it doesn't happen again. ®
Regcast training : Hyper-V 3.0, VM high availability and disaster recovery
COMMENTS
New O2 Tariff
I'm on the new O2 plan... Unlimited Smoke Signals 250 Pigeons a month Free messages in a bottle to other O2 users
Re: Is this the start of a trend?
I hate to say that this is not the start but more the culmination of a pattern of companies letting accountants make decisions on things they dont understand.
It generally works like this, the network architect comes up with a plan which minimises single points of failure, has built in redundancy and lots of expensive shiny, shiny added to it so its bombproof, bulletproof and tea lady proof.
This then goes to the accountants who skim over the reasons why you need this, look at the final price, reduce it by 50% and then pass it back for "remodelling" so that it fits the budget as opposed to the requirements.
Consequently all the bits that you need for a disaster are removed as the accountants dont see the relevance until the disaster happens.
Outsourcing is a classic example of this and RBs showed exactly what happens when you cut corners. Its fine whilst it works until it goes wrong and only then are the lessons previously taught taken seriously, but by then it is too late.
What went wrong
They spent all the money on senior manager bonuses and ignored the single points of failure in the network.
That should save O2 making something up in a reply to you.

IT infrastructure monitoring strategies
Agentless Backup is Not a Myth
Steps to Take Before Choosing a Business Continuity Partner
Requirements Checklist for Choosing a Cloud Backup and Recovery Service Provider
Data control in the cloud