How City IT is under attack from politicians, diesel bugs, HR
Oh, and the stock exchange could blow any moment....
Secret source, security and BYOD
Pic credit: Kodomut, Flickr
Banks have been pretty resistant to the idea of BYOD beyond letting people use their own smartphones and most (but not all) have got it into their heads that unlocked USB ports are accidents pausing to choose which place to happen.
But then again investment banks face different threats to their country cousins in retail. For example, Goldman Sachs is still concerned about how Sergey Aleynikov made off with some of its source code. Aleynikov was convicted in 2010 in federal court but freed after one year in prison by an appeals court. But New York state prosecutors last year re-charged the former Goldman coder with illegally copying computer software. The programmer has pleaded not guilty.
Most data and source code at a firm isn’t even interesting to the people who work there, much less anyone else and the dullest part of my expert witness work is going though people’s email. However drab my life sometimes feels, it is joy compared to many. The problem is that if you make it hard to get at the source, then when something bad happens, the reaction is slow.
Because important code can be deliciously complex, coming to it cold takes up valuable time and increases the chance that your fix will make it worse and I can share that recent incidents, both public and kept private have made most banks and especially hedge funds apply much tighter controls over source code that previously was left on open access servers and shared freely around the firm. You can’t eliminate risk here, just choose one which will hurt least.
Data storage is skyrocketing, not just because of big data analytics but because regulators want a lot more records kept and for longer, which means previous capacity planning is often wrong and occasionally spectacularly so, like when one IT manager found he was 140TB short of what he needed now and Hitachi had to get the gear on site within 24 hours.
Good suppliers can do this, but you’d be a mug to plan it that way and a quiet casualty of the banking recession is that the supporting ecosystem of specialist tech vendors have been cut to the bone or have gone under. This has removed a crutch for many IT managers including myself and leaning on a crutch that isn’t there may look funny, but trust me it isn’t.
Because banks rarely skimp on network hardware when it is installed. IT leaders have found it an easy and painless way to make a clear saving, at least on paper since they can point to the money not spent without denying a new service to anyone important. In many cases this has actually led to an improvement in reliability since upgrades always have a few snags and the decent quality physical equipment is mostly in the nice bit of the reliability curve between bedding in and getting old.
But the banking recession has dragged on a bit and where this coincided with a timely infrastructure upgrade, there is weakness, especially in the midscale firms. Huawei is a subject of much debate, the kit is attractively priced and seems to do the job and although Cisco might sometimes act with, er, "unmitigated gall", its misbehaviour remains within bounds. Meanwhile, Huawei’s alleged shadowy links to the Chinese military and its hackers frightens a lot of IT management. But smaller firms will see the prices and be tempted - and yes I’m aware I may be falling for Cisco PR here.
Disaster recovery is better, almost good, because for once the regulators seem to have done their job on the structurally important firms, requiring them to treat DR as less of a joke and burden than previously, most are now not only properly equipped, but also tested. Almost gone are the times when “business continuity” relied upon hardware demoted from front line use and of questionable reliability and performance. But a critical weakness for them in the past and for smaller firms still is that “testing” was done by the technologists at weekends.
This means that the systems look OK to people whose jobs are to build them but not to use them, with the results that are sometimes very nearly funny. Of course the spreadsheet that no one in front office bothered to tell central IT about doesn’t get replicated. And the devs were lucky to discover in one particular test that none of the traders could log in, because IT had tested it as themselves with supervisor access.
The weakness in all of these cases is that the big boys usually still assume that IT pros are from the planet Krypton and thus cannot be stopped by terrorism - or even just late-running trains on the London Underground - with too little effort being applied to multi-skilling people to cover for those who don’t make it to the DR site.
Threats to you
Compliance now has teeth, or at least the regulators are holding their mouths closer to your groin. Compliance managers tell me that the steer they are getting is that the measure of a good compliance regime is no longer a lack of reported incidents but rather the punishment for those who get caught.
This includes the whole bank. In the good old days, traders worked out the boundaries of what was acceptable by pushing until someone gently pushed them back, this being hardly an IT problem at all. In fact if an IT pro reckoned there was something dodgy going on, he’d be reckless with his career if he made any sort of fuss.
At this point I could mention the ability of the regulators to protect whistleblowers but we both know I’d be taking the piss if I did anything other than sneer at the idea. By all means write an innocuous email that cites “some possible issues we need to address at someone point” to try to cover your back, which is at most better than nothing.
The neglect of the UK’s infrastructure may not be the fault of City IT managers, but it certainly will be their problem. Not only have we gone beyond the point where we can avoid power cuts due to lack of generating capacity but even the cables and transformers that get it there are so badly maintained or overloaded that we are seeing explosions both under pavements and in their housings. One of the larger transformer buildings in the heart of the City almost next to where the Stock Exchange servers live is surprisingly warm to the touch. Yes that’s a lot of power, and no I’m not saying where.
The world’s major governments pander to farmers by using a mix of sticks and carrots to persuade fuel users to burn food for power or transport in the form of biodiesel. Given that most that go hungry aren’t even slightly white and their farmers are usually pink, this ain’t going to change any time soon and of course the earnest arts graduate who heads up “corporate responsibility” at your bank will lever IT into fuelling standby generators with biodiesel.
Without bugging you with too much organic chemistry, the short version is that if you leave oil in a can for a million years, it doesn’t change much. It's obvious really when you think about how much of it has survived. But biodiesel rots. Bugs live in it and unless you are a complete evangelical it is reasonable to suspect that ever more bugs are evolving to feed upon it.
Bugs don’t burn. Yes OK they do, but not well and certainly not in the fine aerosols that you get in diesel engines, which means the engines block up. That’s less of an issue in a car, since if it bungs up, you can get it fixed, a hassle but nothing more. That’s not the case with a data centre power supply, you need it *now* and the batteries won’t keep things going for long enough to get it cleaned out.
I didn’t mention the site above because there is increasing concern about hacktivists, DDoS attacks et al, even though so far this has amounted to little more than defacing the occasional website which is further away from critical systems than the coffee machines that fuel their IT pros.
In summary, I can’t tell you what will blow up next, but it is a system under strain with regulators driven by politicians whose grasp of any IT rarely reaches the giddy heights of owning an iPad - and where some serious money is handled by systems that would be scoffed at by a midsized supermarket chain. ®
Dominic Connor is a City headhunter, former CIO and before that a grunt programmer at various banks. You can connect to him here.
Sponsored: Hyper-scale data management