Brit cloud slinger iomart goes TITSUP, knackers Virgin Trains, Parentpay

Updated Brit cloud provider Iomart is having an outage-ridden day - but its service-outage-hit customers are arguably having a far worse one.

Reports first began to surface on Twitter at 12:26 this afternoon with users of Iomart subsidary Melbourne IT noting some problems.

@MelbourneHost @iomart Problems??

— Nutty About Hosting (@nahosting) March 29, 2018

20 long, long minutes later, Iomart confirmed that something very bad had indeed happened

Apologies - we are currently experiencing some network issues. Our team is investigating and we will update you as soon as we can.

— iomart (@iomart) March 29, 2018

Finally, at 1:09pm, the cloud slinger confirmed what some users already knew: three data centres, in Glasgow, Edinburgh and Manchester, had gone for a unscheduled nap thanks to a break in the fibre network.

Two simultaneous breaks in the firm's fibre network occurred, one south of Manchester and another between Glasgow and Nottingham.

The cause of the fibre borkage is unknown at this point, but one customer asked if the outage was the result of "a synchronized attack by Geordie and Cheshire Moles" or if was simply that "@iomart are useless?"

It is now plain this is a synchronized attack by Geordie and Cheshire Moles. Alternatively @iomart are useless?

— Alexander B (@CandlemasReiver) March 29, 2018

There is no word on how these breakages could have caused the outages, nor what disaster recovery plans are in place. Neither is there any word on when normal service will be resumed. Iomart's status page reports things are still down at time of writing.

Iomart, which sells services to both the private and public sector, has been subject to a barrage of angry tweets from users suddenly finding themselves without critical services.

Very disappointed now - over an hour down and no communication from you @iomart. You surely know by now how long this might take to resolve.

— A Digital. Agency (@adigital_uk) March 29, 2018

Can you give us the number for your compensation line, we're losing absolutely thousands with this downtime

— Unique Fundraising (@UniqueFundraisi) March 29, 2018

The outage has hit organisations thoughout the UK, with Virgin Trains East Coast customers facing worse than normal service thanks to being unable to book or collect tickets due to the failure.

Looks like we've ran into some IT issues this afternoon! If you're having trouble booking or collecting tickets, or accessing tickets through the app, have no fear - the Web Wizards are on the case!

Apologies for any inconvenience caused. pic.twitter.com/SJMNBuROSF

— Virgin Trains EC (@Virgin_TrainsEC) March 29, 2018

Digital school food dispenser app Parentpay, which appears to host its infrastructure with Iomart, has also gone down as a result of the outage, with furious parents demanding to know how little Joffrey is meant to pay for his boiled cabbage and mash.

The feed-your-kid app (corporate tagline: “Count on us”) allows parents to send money to schools for meals without entrusting it to their beloved offspring’s pockets.

One aggrieved user posted on Parentpay’s Facebook page, after the firm chose to tell world+dog of the outage on there: “As a School Parentpay administrator I haven't seen an email from you with this information on it. We have had to add a loan facility onto student accounts to enable students to eat today as parents have been unable to top up this morning. Please can you advise who would received the email you sent in a school?”

So parent pay can update it's website to announce that it can't let you log on but it can't let you log in? Sitting an A level history exam on an empty stomach will be fun thanks guys

— Scrappy littlenobody (@elliefred123) March 29, 2018

A Parentpay spokeswoman told us: “We experienced technical difficulties today that meant users were unable to access the service. We have identified the issue and all technical problems have now been resolved. We informed all schools of the issue before the lunch period with a view to ensuring that no child went without lunch."

“We’re really sorry and please accept our apologies for any inconvenience caused. We will be providing a more detailed update to our customers in due course,” she added.

Iomart may wish to reconsider some of its recent sniping at Amazon’s AWS in a blog post, or tweets such as the one below, which appeared mere hours before the outage and has been captured by The Register for posterity.

The firm reported revenues of £89.6m in 2017, up 17 per cent on the previous year. Spending some more of those gains on its infrastructure may be a good idea in the light of today’s events.

The Register has asked iomart to comment. In a statement sent to us at quarter past 5pm, it blamed a third party supplier for a cable fault. ®

Updated to add

We've learned iomart is carrying out maintenance on one of its UK data centers at the weekend.

"As part of iomart's ongoing commitment to continual improvement and reliability, the iomart network team will be conducting scheduled maintenance to upgrade the deployed software version in use on the Cisco Application-Centric Infrastructure (ACI) network fabric within our Maidenhead (DC5) datacentre," a note to customers this month, seen by El Reg, reads.

"The maintenance work will begin at 23:00 (BST) on Saturday 31st March 2018 and last for 8 hours until 07:00 (BST) on Sunday 1st April 2018.

"As with any upgrade or maintenance work there is an increased risk, however this upgrade does not require any customer downtime. The iomart network operations centre in conjunction with the iomart network team will be monitoring all customer services throughout the window and following the completion of the upgrade works."

Fingers crossed, eh!

PS: TITSUP – Teetering Infrastructure Turns Services Utterly Pants