Cisco thanks customers after website outage
Mistake leads to two hour outage
SaaS data loss: The problem you didn’t know you had
Cisco is keeping schtum on how exactly it contrived to down its own website worldwide for over two hours yesterday.
Customers and other web browsers notice something was amiss yesterday morning UK time as the network giant's web presence completely evaporated, without any warning.
A Cisco statement said: "Cisco can confirm that www.cisco.com experienced an outage of just over 2 hours this morning/afternoon. This was an inadvertent result of a change made during regular maintenance - a change that has since been reversed returning www.cisco.com to normal service."
It added, "We thank our customers and partners for their patience and apologize for any inconvenience they may have experienced."
But the vendor is not saying which piece of equipment was being maintained and how an error in the maintenance procedure caused unforseen results which cascaded throughout its website infrastructure and brought it down for more than two hours.
This is a fine example of a failure in business continuity and disaster recovery by the world's leading networking supplier.
All the backup and alternative data centres across Cisco's entire global estate, and all the BC/DR processes in place were of no use at all when someone, a Cisco employee possibly, made a mistake during regular maintenance. Clearly Cisco's business continuity and disaster recovery testing scenarios had not included the event that caused the outage.
The company is not saying how much the outage cost in terms of lost e-commerce. With a site of such prominence we could well imagine it is substantial. ®
COMMENTS
Cisco's sales pitch rings hollow
When you are buying in ones and twos then yes, you use a distributor. But when you are a big customer buying by the pallet the distributor just gets in the way. But a big customer's order isn't going to be put off by a two hour outage (assuming it isn't the last two hours of the financial year).
The problem for Cisco is one of perception. Cisco make a lot of money by telling people to build an all-Cisco network to get the ultimate in reliability, and when the prime example of an all-Cisco network goes down hard -- and obviously can't be easily rescued -- then that sales pitch starts to sound empty.
#1 cause of outages is change
People making changes is the #1 cause of network outages, as confirmed by pretty much all IT research firms.
It shouldn't be a surprise that a single change caused this outage.
http://www.netcordia.com/solutions/automate-network-change.asp
They forgot to buy the extended support agreement.
Someone must have forgot to spend the extra $$$ to purchase Cisco's maintenance and support. They probably couldn't get anything fixed without it.
IT infrastructure monitoring strategies
Agentless Backup is Not a Myth
Top 10 SIEM implementer’s checklist
Steps to Take Before Choosing a Business Continuity Partner
Enabling efficient data center monitoring
