Telstra hauls in Cisco, Ericsson, Juniper to explain TITSUPs
Australia's dominant carrier wants suppliers to explain why its network keeps going down
Australia's dominant carrier Telstra has hauled three of its key vendors into the principal's office for a dressing down, following network outages that had the Australian incumbent giving mobile users “free data days” to stem the outrage.
Major outages in the mobile network happened in February and March, and the carrier's network status page still shows parts of the network that remain TITSUP (Total Inability to Support Usual Performance).
Kate McKenzie, Telstra's COO, told the Comms Day Summit that the network stumbles in February and March have led the company to launch a review of its networks.
That project will involve three vendors: Ericsson, which dominates its mobile infrastructure; Cisco, the carrier's core IP routing supplier practically forever, and Juniper Networks.
“In conjunction with our global partners Ericsson, Cisco and Juniper we have assembled a team of internal and external engineering experts to do an end-to-end review of our network. While this work is underway, Telstra Operations has a heightened awareness plan including Executive-level review of any changes planned for the mobile and core IP networks”, she said in the speech (in full, here).
McKenzie explained to the summit that the February 9th outage was (as previously reported) down to human error while engineers were trying to isolate a degrading node from the network.
The incorrect restart of a node dropped 15 per cent of mobile customers from the network at once, and with them all trying to re-register at once, the network collapsed.
The second outage, on March 17, gets only a partial explanation. For reasons not explained in the speech, “initially international roaming customers, and then domestic customers as well” were unexpectedly disconnected from the network, and the registration database couldn't cope when they all tried to reconnect.
“Our industry experts have already told us that this is a global challenge faced by many in the industry”, McKenzie says.
Number three was the SIP outage that happened on March 22, which she says affected only three per cent of customers in total, across mobile, IP telephony, and NBN voice services.
McKenzie says she is leading the network review, and that the carrier's Home Location Register (HLR, responsible for managing mobile devices' logins to base stations) will get new capacity “within a few days”.
Customers still suffering TITSUPs at Telstra
Both unexplained and as-yet-unresolved is the persistent connection loss happening on Telstra's ADSL and Wi-Fi services. Customers are still suffering repeated loss of their IP addresses, interrupting their Internet access. ®