Cloud based data management

Analysis Banks need to start thinking of themselves as IT companies, said David Chan of City University London.

"A senior banking technologist has said to me: 'A retail bank is nothing but an IT company with a banking licence'," Chan told The Reg. "While this may seem extreme, when one looks at the economics of any retail bank, it is clear that this is the case."

While Chan considers it unlikely that we'll see another catastrophe on the scale of the RBS meltdown, he does see the recent spate of technical cock-ups as evidence that the cut and cut-some-more approach to costs over the past two years, added to underinvestment during the past 20, is starting to hit customer-facing services. It's a pattern we may be familiar with, he explained:

When Sony got hacked, I imagine the guys in Playstation security had been saying for years [that] 'We need to invest more in this'. Or at Microsoft in the 1990s, I'm sure there were guys saying, 'We have got to spend more in patching holes in Windows', and senior management were telling them to add new features.

And with people shifting banks post screw-up, IT performance is having an impact on customer loyalty for almost the first time in decades.

What RBS really got wrong: its mentality

When RBS CEO Stephen Hester faced a committee of MPs in the aftermath of the banking disaster, he said what The Reg had pieced together at the time - that the root cause of the problem was "a routine software upgrade managed and operated by our team in Edinburgh, which caused the automated batch processing software [CA-7] to malfunction".

The Royal Transaction Processing
Company of Scotland

But Chan, who is the director of his university's Centre for Information Leadership, suggested the bank's catastrophe goes beyond a piece of software or one butter-fingered operative: he zeroed in on organisational mentality as the longterm systemic cause of RBS's woes.

A routine procedure shouldn't kill a bank's core operation for three weeks, and the actions of one employee or team of employees shouldn't affect 17 million customers, he said:

I think the fact that batch scheduling uses CA-7 is irrelevant to the root-cause of this particular failure. People do make mistakes but the processes that were in place and the expertise in place had been diluted. The RBS problem is ultimately symptomatic of a culture within many large organisation where the senior team believe that they can rule by “dictat” without having to understand the consequences of their decisions.

And this is what makes offshoring to Chennai, in south east India, a problem: although there is nothing intrinsically wrong with it, out-sourcing can exacerbate bad management. "There's more of a bureaucratic culture in India," said Chan:

This isn't true of everyone but in general engineers over there are more likely to do what the boss says: "Ok, the boss wants something cut, so we'll cut it." This is opposed to coming back to him and arguing against a decision.

That means that when management makes a bad call, there is less chance the executives will be challenged and more chance their boardroom edicts will be carried out.

"If you hire IT people in a support service function, you get a support service mentality," Chan said.

The cost of the cuts: deskill your workforce at your peril

And it's not just a culture issue: there's a trend towards pigeonholing folk into particular roles within a process, ensuring that they do one or two things well and need not worry about the rest of the machine. It's an approach that's at odds with old-school IT pros who were expected to know the tricks and every turning cog of an entire system. Chan said:

RBS staff had always had a fair number of people working in all functions who had a holistic view of the processing. This not only resided in the architectural teams but also in operational teams. Traditionally, development teams were responsible for delivering a transition plan with roll-back procedures included. Operational staff had enough experience to challenge such plans and ensured that these plans were robust.

One well-placed source close to the banking industry told us: "Systems programming has become a little de-skilled in recent years, since software from IBM and third-party vendors (for example, CA) usually just works. They are black boxes now, so there is little ability to make a real-risk assessment, unless you do expensive things like parallel runs for a few nights to compare and contrast."

Our mole reiterated Chan's point: deskilled bods working to a set of written procedures can work fine 99 per cent of the time, but during that one per cent when things go wrong, they won't know what to do and will exacerbate any delays. If the guy who spots a problem doesn't know who to approach, recovery is stalled, which causes more problems in a moment of crisis. Our source said:

This is where institutional knowledge comes in. No matter how good your written procedures, something which escalates up and down hierarchies of people to be notified is not going to work quickly.

Ensure Ease of Recovery with Asigra’s Agentless Software

Next page: So. How can they stop that meltdown from happening again?