RBS must realise it's just an IT biz with a banking licence
Expert drills into what it'll take to prevent another bank technology fiasco
Analysis Banks need to start thinking of themselves as IT companies, said David Chan of City University London.
"A senior banking technologist has said to me: 'A retail bank is nothing but an IT company with a banking licence'," Chan told The Reg. "While this may seem extreme, when one looks at the economics of any retail bank, it is clear that this is the case."
But it was IT that tripped up financial institutions this summer: a high-profile system crash left 17 million customers of Royal Bank of Scotland (RBS), including its Natwest and Ulster Bank components, without access to their money for a week in July - three weeks in some cases. Two debit card-related screw-ups happened last week at building society Nationwide and at RBS-owned Natwest.
While Chan considers it unlikely that we'll see another catastrophe on the scale of the RBS meltdown, he does see the recent spate of technical cock-ups as evidence that the cut and cut-some-more approach to costs over the past two years, added to underinvestment during the past 20, is starting to hit customer-facing services. It's a pattern we may be familiar with, he explained:
When Sony got hacked, I imagine the guys in Playstation security had been saying for years [that] 'We need to invest more in this'. Or at Microsoft in the 1990s, I'm sure there were guys saying, 'We have got to spend more in patching holes in Windows', and senior management were telling them to add new features.
And with people shifting banks post screw-up, IT performance is having an impact on customer loyalty for almost the first time in decades.
What RBS really got wrong: its mentality
When RBS CEO Stephen Hester faced a committee of MPs in the aftermath of the banking disaster, he said what The Reg had pieced together at the time - that the root cause of the problem was "a routine software upgrade managed and operated by our team in Edinburgh, which caused the automated batch processing software [CA-7] to malfunction".
The Royal Transaction Processing
Company of Scotland
But Chan, who is the director of his university's Centre for Information Leadership, suggested the bank's catastrophe goes beyond a piece of software or one butter-fingered operative: he zeroed in on organisational mentality as the longterm systemic cause of RBS's woes.
A routine procedure shouldn't kill a bank's core operation for three weeks, and the actions of one employee or team of employees shouldn't affect 17 million customers, he said:
I think the fact that batch scheduling uses CA-7 is irrelevant to the root-cause of this particular failure. People do make mistakes but the processes that were in place and the expertise in place had been diluted. The RBS problem is ultimately symptomatic of a culture within many large organisation where the senior team believe that they can rule by “dictat” without having to understand the consequences of their decisions.
And this is what makes offshoring to Chennai, in south east India, a problem: although there is nothing intrinsically wrong with it, out-sourcing can exacerbate bad management. "There's more of a bureaucratic culture in India," said Chan:
This isn't true of everyone but in general engineers over there are more likely to do what the boss says: "Ok, the boss wants something cut, so we'll cut it." This is opposed to coming back to him and arguing against a decision.
That means that when management makes a bad call, there is less chance the executives will be challenged and more chance their boardroom edicts will be carried out.
"If you hire IT people in a support service function, you get a support service mentality," Chan said.
The cost of the cuts: deskill your workforce at your peril
And it's not just a culture issue: there's a trend towards pigeonholing folk into particular roles within a process, ensuring that they do one or two things well and need not worry about the rest of the machine. It's an approach that's at odds with old-school IT pros who were expected to know the tricks and every turning cog of an entire system. Chan said:
RBS staff had always had a fair number of people working in all functions who had a holistic view of the processing. This not only resided in the architectural teams but also in operational teams. Traditionally, development teams were responsible for delivering a transition plan with roll-back procedures included. Operational staff had enough experience to challenge such plans and ensured that these plans were robust.
One well-placed source close to the banking industry told us: "Systems programming has become a little de-skilled in recent years, since software from IBM and third-party vendors (for example, CA) usually just works. They are black boxes now, so there is little ability to make a real-risk assessment, unless you do expensive things like parallel runs for a few nights to compare and contrast."
Our mole reiterated Chan's point: deskilled bods working to a set of written procedures can work fine 99 per cent of the time, but during that one per cent when things go wrong, they won't know what to do and will exacerbate any delays. If the guy who spots a problem doesn't know who to approach, recovery is stalled, which causes more problems in a moment of crisis. Our source said:
This is where institutional knowledge comes in. No matter how good your written procedures, something which escalates up and down hierarchies of people to be notified is not going to work quickly.
Mr Chan, has in my opinion, hit the proverbial nail on the head.
The middle layer of management never seems to get mentioned, it isn't Mr Hester that is choosing to take risks by de-skilling the workforce but it will be his targets that are causing the problem. Required reduction in IT costs = Management finding an easy fix = ship the development off to a cheaper country whilst retaining their own jobs/salary/bonus.
It is the middle management layer that has failed. They chose not to listen, they ploughed ahead with changes and it has cost the company dearly. Given the culture at RBS, no one questions their boss above project manager level. If you did then the end of year rating system was used as a big stick.
I have said it before but the outcome of any review will be "more control". It wont point the finger of blame at any management strategy, it will be just an extra layer of change control that slows the development process down even further.
> trend towards pigeonholing folk
> And it's not just a culture issue: there's a trend towards pigeonholing folk into particular roles
> within a process, ensuring that they do one or two things well and need not worry about the
> rest of the machine.
Ain't that the truth... I was recently told by a line manager "we don't need people like you who understand everything any more". Well, next time the proverbial hits the fan I almost certainly won't be there (hence icon - would you given a hint like that) and I wonder how well they'll get on...
CM is a communications tool. It works if it puts the change plans in front of people who are technically and institutionally able to challenge them. It works better if those people include the ones who will have to fix what goes wrong. What can't work is a lovely paper trail with all the manager's signatures.
Re: Good article
The tech staff in Edinburgh are competent and know what needs doing (at least, they were a few years back) but the middle managers are handcuffed by terror of the VPs. Any question or criticism of those above you is a serious CLM.
Many of the technical staff are contractors who have little vested interest in the job beyond this weeks pay cheque. (Admittedly, a very generous one.) I asked a colleague why so many critical systems ran on failure-prone, support-intensive MS Windows systems. He replied with a shrug "We get paid by the hour."
Re: "A (retail) bank's business is Risk Management."
"3) A retail bank has no reason to put any of its customers money at risk."
Rubbish. A retail bank takes in customer deposits at one rate and lends them out at a higher rate, the difference being their profit. Even if we ignore the fact that they "create" money by lending out more than they have on deposit there is always going to be a risk involved in lending out.