Feeds

How spreadsheets (nearly) conquered and killed the financial industry

Would you trust millions on an Excel formula?

Protecting users from Firesheep and other Sidejacking attacks with SSL

Step forward the spreadsheet, saviour of those chained to change requests

Though there seemed to be no hope of having this process simplified, there was a tacit recognition by management that it was completely unworkable in the dynamic and fast-moving world of spreadsheet development by “rapid deployment” teams working closely with the business. (I managed one of these teams, with members in London, New York, and Hong Kong, each supporting their local desk.)

For these developers, unlike the teams who supported larger-scale, Linux-and-Java-based systems, the usual rules simply didn’t apply. Spreadsheet changes could be pushed out at will, and at a moment’s notice.

In addition to removing a tremendous amount of drudgery and bureaucracy from these developers’ lives, this across-the-board exemption made it much easier for traders to get their day-to-day work done, since their requests could be implemented almost immediately.

This resulted in a kind of positive feedback loop, where desk developers began to be seen as the only ones who could deliver usable systems when they were needed.

But while it was true that development of large systems was tortuously slow and horribly error-prone, it was clear to anyone working in technology that there were many reasons for this divergence: in addition to being burdened with the tedious rollout procedure described above, teams tasked with building industrial-strength systems suffered from top-heavy management, uncooperative system administrators and support staff (one of many symptoms of an out-of-control bureaucracy), and a distance from the desk that proved disastrous — all in sharp contrast to the rapid deployment teams, who sat next to the traders all day and are were intimately familiar with their workflows.

Furthermore, the design and coding of a Java system is always significantly more involved than relatively simple Excel scripting, making it easy for people doing the latter to become instant heroes.

Excel with Excel

All these factors led to an ever-increasing reliance on rapid-deployment teams and spreadsheets. Large development projects either failed outright or couldn’t be launched to begin with because they needed big teams and long-term horizons for the design, development, and QA.

So the desk frequently went with the “stopgap” of a spreadsheet.

This would invariably be seen as tactical (i.e., short-term), but again and again, when the more permanent system never arrived, the spreadsheet became the de facto strategic solution, much easier than trying to address the bigger problems that were causing so many large-scale development projects to falter. As a result, many processes that were plainly too big or complex to run in Excel over the long term did.

The result: frequent memory blowouts, painfully slow calculation cycles, and PCs pushed past way their limit. With piles of money riding on the deals that were being priced, or the risks that were being hedged, these seemingly minor PC hiccups had the potential to cause not-at-all-minor losses.

But the temptation to go with the quick solution was too great, and large-scale development too trouble-plagued, for most people to look at this state of affairs objectively.

While the example I’ve just described was the most extreme I’ve seen, the tendency to reach for the quick and dirty solution of an Excel spreadsheet whether appropriate or not is common on trading desks everywhere. Traders like the idea of having direct control of their processes, or at least being able to see their workings; large-scale systems become harder to design and build as they get more complex, and the development teams harder to manage; and rapid-deployment teams are generally free from the burdensome rules that other developers have to adhere to, making them more agile in their response to fast-changing business requirements.

But this short-sightedness can lead to a variety of problems: relying on lightweight, desktop-based solutions for business-critical functions; giving too much power to users in situations that require centrally managed checks and balances; and circumventing the professional monitoring and full-time production support that naturally come with large-scale systems.

Giving users too much control and flexibility can be a very bad idea when the processes involved are especially delicate, or where the amount of money at stake is large enough to render even an innocent mistake disastrous. ®

Website security in corporate America

More from The Register

next story
Hey, Scots. Microsoft's Bing thinks you'll vote NO to independence
World's top Google-finding website calls it for the UK
Phones 4u slips into administration after EE cuts ties with Brit mobe retailer
More than 5,500 jobs could be axed if rescue mission fails
Apple CEO Tim Cook: TV is TERRIBLE and stuck in the 1970s
The iKing thinks telly is far too fiddly and ugly – basically, iTunes
Israeli spies rebel over mass-snooping on innocent Palestinians
'Disciplinary treatment will be sharp and clear' vow spy-chiefs
Huawei ditches new Windows Phone mobe plans, blames poor sales
Giganto mobe firm slams door shut on Microsoft. OH DEAR
Phones 4u website DIES as wounded mobe retailer struggles to stay above water
Founder blames 'ruthless network partners' for implosion
Found inside ISIS terror chap's laptop: CELINE DION tunes
REPORT: Stash of terrorist material found in Syria Dell box
OECD lashes out at tax avoiding globocorps' location-flipping antics
You hear that, Amazon, Google, Microsoft et al?
prev story

Whitepapers

Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.
Security and trust: The backbone of doing business over the internet
Explores the current state of website security and the contributions Symantec is making to help organizations protect critical data and build trust with customers.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.