The Register® — Biting the hand that feeds IT

Comments on: Royal Mail sites hit by downtime cock-up

Who run the Royal Mail website I wonder 

Posted Friday 4th April 2008 12:13 GMT

Ah, the favicon on royalmail.com is a bit of a giveaway...

Smartstamp 

Posted Friday 4th April 2008 12:21 GMT

Their Smartstamp servers were up until around 11:00 then went down to join the rest of Royal Mail. Looks like it wasn't a big-bang failure then but they've taken everything else down, just in case.

So... 

Posted Friday 4th April 2008 12:22 GMT

Coat

their web page is in the post then?

"I couldn’t possibly speculate without being in possession of all the facts" 

Posted Friday 4th April 2008 12:34 GMT

Joke

That's fine, please get all the facts, and THEN speculate for us.

site down time 

Posted Friday 4th April 2008 12:35 GMT

The Royal Mail site has been downed by a piece of faulty hardware in a date centre in copenhagen, RM's IT provider CSC is ment to be working on the issue.

Simon... 

Posted Friday 4th April 2008 12:37 GMT

Simon, who's icon IS that? I have seen it before but can't put a name to it!?

Thanks,

Dave

It'll be up in 24 hours 

Posted Friday 4th April 2008 13:02 GMT

If they tell you it'll be up by 9:30 tomorrow morning, you can actually expect it at 14:45 on tuesday.

gotta say though 

Posted Friday 4th April 2008 13:10 GMT

i was dreading using the phone to trace a parcel - i figured i'd be on hold for hours - but it only took 5 minutes of "please press 1 now, otherwise hold the line" before i got to speak to someone

they weren't able to find my parcel either, but that's besides the point - the phone did prove to be a viable fallback device

Fun place to work at RoyalMail? 

Posted Friday 4th April 2008 13:23 GMT

Joke

Like who exactly are;

Philatelic Customer Services and Smilers Customer Services!??

Sound like new age funky departments to be in!

FAO Dave 

Posted Friday 4th April 2008 13:23 GMT

Happy

http://www.sun.com/

@Kenny Millar 

Posted Friday 4th April 2008 13:25 GMT

Stop

It might be up in 24 hours, but don't forget that you'll have to take the card, and some id, to the nearest sorting office, between the hours of 7:00am and midday, on a normal working day, and pick it up yourself...

Bad 'time of the month' for Xansa.. 

Posted Friday 4th April 2008 13:28 GMT

Coat

.. or F.I. as they used be called.

Funny that 

Posted Friday 4th April 2008 13:37 GMT

Today is the e-business leaving do

"We apologise for any inconvenience" 

Posted Friday 4th April 2008 13:58 GMT

I hate that phrase. "Any"?

untitled - or just no postcode 

Posted Friday 4th April 2008 14:04 GMT

Thumb Up

"A spokesman told us that Royal Mail tech bods are investigating the problem, but was extremely vague when asked to provide further details"... well it's like a sort of problem thingie and it's proving to be problematic to us because, err... well... it's causing a bit of a problem, especially to our, um, problem solvers.

<wonders> did anyone check to see if the cleaners put the plug back in?</wonders>

@anonymous coward 

Posted Friday 4th April 2008 14:12 GMT

What sort of crap excuse is that? Whatever happened to redundancy, clustering, virtualisation? A hardware failure shouldn't bring down an important service in this day and age.

Not been right for 2 weeks 

Posted Friday 4th April 2008 14:22 GMT

Black Helicopters

I have sent a dozen items in the last fortnight by Recorded Delivery or Special Delivery. None of the tracking numbers have appeared on their website (but luckily have been delivered).

it's back now 

Posted Friday 4th April 2008 14:30 GMT

Happy

yay I can finally find out if my local branch is open tomorrow morning!

@KarlTh - It's not an excuse... 

Posted Friday 4th April 2008 14:32 GMT

...it's just an explanation. I suspect our AC entirely agrees with you, but he's (or she's) an AC for a good reason.

Bean count 

Posted Friday 4th April 2008 15:05 GMT

Boffin

@ KarlTh

the way it works is, the primary may be big and beefy, but the secondary is less than 50% the spec on everything, starting with memory. the accountants always want a word when that failover box is purchased, as an idle system already gets their thongs in a knot, and a well-specced one is simply inconceivable (credit: Princess Bride). all one can do, as the original system scales up to 90% or better utilization, is:

[1] put forward a funding request for an adequate failover system every year (it is, of course, rejected, or deferred until next year), and/or

[2] bring up a JBOC* (built from spare parts and scavenged components) tertiary that would take some of the load, but is unproven, unsupported, unauthorized, and off the books (management will force IT to ditch it if they find it, and the persons responsible may lose their jobs), and/or

[3] pray (this will not work, but makes everybody feel better, until the primary "falls over and catches on fire").

odds are, CSC are replacing the failed components in the primary, and possibly doing a bare-metal restore (assuming they have a backup). it is also possible that the primary's UPS had a catastrophic failure (power "flapping", or similar), that may have hosed the primary system quite thoroughly, without actually baking anything. in any case, it all sounds exceedingly unpleasant, and the setup was probably planned with a good measure of delusional optimism.

to conclude, this happens, far too often, and is in most cases a management failure, but occasionally a design issue, and very rarely a technical fault (assuming that equipment has a finite lifespan, which all equipment does, component failure is part of the normal scope of events).

*Just a Box Of Components

Collect in 48 hours 

Posted Friday 4th April 2008 15:17 GMT

"Apologies, we were unable to deliver this webpage to you today. As a result the webteam will have stored it overnight in the boot of their car and will return it to the sorting office sometime tommorrow. Please collect it from there after 48 hours ... please bring a form of identification for you for the IP address to which the page was meant to be delivered"

@ Flatspot 

Posted Friday 4th April 2008 15:58 GMT

FYI, Philatelic services are to do with stamp collecting and Smilers are a particular type of stamp. The first term has been around for years but the second; well, it probably did come fromt the whale song department.

Should have gone to Zeus 

Posted Friday 4th April 2008 16:05 GMT

IT Angle

http://www.zeus.com/

Speculation? 

Posted Friday 4th April 2008 16:32 GMT

Joke

It's probably just a glitch as they test the new profiling system.

Redundancy 

Posted Friday 4th April 2008 17:59 GMT

If this is a failure of a piece of infrastructure equipment, then as far as I'm concerned the muppets get what they deserve. How stupid to have a single point of failure in their infrastructure. But what have we come to expect over the years from Royal Mail..incompetence.

Doesn't look as if they have heard of fault tolerance does it?

And as for secondary systems only being half the spec of the primary..kinda defeats the point doesn't it, one would expect the back-up system to provide the same level of service, which obviously means the same spec equipment!

@b shubin 

Posted Friday 4th April 2008 22:45 GMT

Yeah, but the secondary system should still work, if slowly and prone to timing out. This is completely down; it looks like there is a single point of failure and it's hosed.

Blame it On ! 

Posted Saturday 5th April 2008 07:22 GMT

Go

Blame it on all that Hot air from the Home Office Wacky Jacky diverting all system resources required for her to implement her version of Police State Web 2.10 !

More on secondary 

Posted Saturday 5th April 2008 12:36 GMT

Pirate

@ KarlTh

you underestimate our fine friends from finance. the secondary is specced at less than 50% on EVERYTHING, including disk space (for which they may have bought a system with 1 big hard drive, if the purchasing is done by the purchasing department and not by IT - not a joke and i've seen this happen). the application may not start, and if the disk got full from replication, the box will hang on boot (if it is a Windows box).

also, depending on what OS it's running, the server may simply drop requests under high load (MS Server 2003 did that when running a commercial Java app server, in a test i've seen).

to conclude, if the secondary is substantially inadequate, it may not boot, or may not be usable if it comes up.

fallback system 

Posted Saturday 5th April 2008 19:45 GMT

"they weren't able to find my parcel either, but that's besides the point - the phone did prove to be a viable fallback device"

well duh - they were checking it on their web based system ...

Hmmm... 

Posted Sunday 6th April 2008 16:52 GMT

Coat

This reminds me of the famous British Rail (snow on the line) excuse. Perhaps the bosses will claim that they would have been okay, except for the fact that online customers were ordering the wrong sort of stamps.

Mine's the one with the self-adhesive back panel.

Maybe... 

Posted Sunday 6th April 2008 17:12 GMT

Alert

this is part of their post office closure program...

Email 

Posted Sunday 6th April 2008 23:54 GMT

Alert

Please email me some stamps, I want to write to someone, oh hang on a minute.....

Deutsche Post's website 

Posted Monday 7th April 2008 07:14 GMT

Coat

I bet this wouldnt happen on the Deutsche Post's website.

They even have a version of the website in English, not bad for johnny foreigner! (probably a cunning trick to discredit the Her Majestys Finest)

just shows you empires are not built on gritty determination, ruthless efficiency and razor sharp tailoring, but by fate, luck,stiff upper lips and putting on a jolly good show?

must blame hardware and bean counters 

Posted Monday 7th April 2008 11:10 GMT

Linux

WE cannot blame winders this time. According to Netcraft all of the listed sites are running Solaris 9/10.

As good as down for weeks 

Posted Monday 7th April 2008 11:32 GMT

Flame

The speed of response of the Royal Mail site has been attrocious recently.

Three minutes to get the second class postage page, fifteen minutes and still no response from the print-your-own postage log-on pages.

As far as I can see there's nowhere on their site to complain about the website - just late deliveries etc. How glad I was when up popped a "Take our Survey" and I was able to feed back...

Also suprised that the Royal Mail traceroute goes from UK to New York to Holland - what no UK servers

So maybe they've taken it down to add a second processor and a few Mb more memory

theres a jolly little song 

Posted Tuesday 8th April 2008 06:58 GMT

Joke

..it ends "C-S-C, k-e-y, m-o-u-s-e!"

Joking aside, don't be in a rush to tar+feather CSC until you know detail of the SLAs in the RM contract... oh alright, you get the tar, I'll pluck the chickens

Don’t Miss

email symbolStill sending naked email? Get your protection here

Security How-to Buckle your seatbelt, encrypt your bits

Google's Satan phoneT-Mobile G1 Google Android-based smartphone

Review Operating System 1, Hardware 0

Ubuntu teaser Ubuntu 8.10 - All Hail new Network Manager

Review The good kind of UI theft

OpenOffice_logoOpenOffice 3.0 - the only option for masochistic Linux users

Review And linear optimizing Mactards