Feeds

How spreadsheets (nearly) conquered and killed the financial industry

Would you trust millions on an Excel formula?

Seven Steps to Software Security

Step forward the spreadsheet, saviour of those chained to change requests

Though there seemed to be no hope of having this process simplified, there was a tacit recognition by management that it was completely unworkable in the dynamic and fast-moving world of spreadsheet development by “rapid deployment” teams working closely with the business. (I managed one of these teams, with members in London, New York, and Hong Kong, each supporting their local desk.)

For these developers, unlike the teams who supported larger-scale, Linux-and-Java-based systems, the usual rules simply didn’t apply. Spreadsheet changes could be pushed out at will, and at a moment’s notice.

In addition to removing a tremendous amount of drudgery and bureaucracy from these developers’ lives, this across-the-board exemption made it much easier for traders to get their day-to-day work done, since their requests could be implemented almost immediately.

This resulted in a kind of positive feedback loop, where desk developers began to be seen as the only ones who could deliver usable systems when they were needed.

But while it was true that development of large systems was tortuously slow and horribly error-prone, it was clear to anyone working in technology that there were many reasons for this divergence: in addition to being burdened with the tedious rollout procedure described above, teams tasked with building industrial-strength systems suffered from top-heavy management, uncooperative system administrators and support staff (one of many symptoms of an out-of-control bureaucracy), and a distance from the desk that proved disastrous — all in sharp contrast to the rapid deployment teams, who sat next to the traders all day and are were intimately familiar with their workflows.

Furthermore, the design and coding of a Java system is always significantly more involved than relatively simple Excel scripting, making it easy for people doing the latter to become instant heroes.

Excel with Excel

All these factors led to an ever-increasing reliance on rapid-deployment teams and spreadsheets. Large development projects either failed outright or couldn’t be launched to begin with because they needed big teams and long-term horizons for the design, development, and QA.

So the desk frequently went with the “stopgap” of a spreadsheet.

This would invariably be seen as tactical (i.e., short-term), but again and again, when the more permanent system never arrived, the spreadsheet became the de facto strategic solution, much easier than trying to address the bigger problems that were causing so many large-scale development projects to falter. As a result, many processes that were plainly too big or complex to run in Excel over the long term did.

The result: frequent memory blowouts, painfully slow calculation cycles, and PCs pushed past way their limit. With piles of money riding on the deals that were being priced, or the risks that were being hedged, these seemingly minor PC hiccups had the potential to cause not-at-all-minor losses.

But the temptation to go with the quick solution was too great, and large-scale development too trouble-plagued, for most people to look at this state of affairs objectively.

While the example I’ve just described was the most extreme I’ve seen, the tendency to reach for the quick and dirty solution of an Excel spreadsheet whether appropriate or not is common on trading desks everywhere. Traders like the idea of having direct control of their processes, or at least being able to see their workings; large-scale systems become harder to design and build as they get more complex, and the development teams harder to manage; and rapid-deployment teams are generally free from the burdensome rules that other developers have to adhere to, making them more agile in their response to fast-changing business requirements.

But this short-sightedness can lead to a variety of problems: relying on lightweight, desktop-based solutions for business-critical functions; giving too much power to users in situations that require centrally managed checks and balances; and circumventing the professional monitoring and full-time production support that naturally come with large-scale systems.

Giving users too much control and flexibility can be a very bad idea when the processes involved are especially delicate, or where the amount of money at stake is large enough to render even an innocent mistake disastrous. ®

Boost IT visibility and business value

More from The Register

next story
Airbus promises Wi-Fi – yay – and 3D movies (meh) in new A330
If the person in front reclines their seat, this could get interesting
BBC goes offline in MASSIVE COCKUP: Stephen Fry partly muzzled
Auntie tight-lipped as major outage rolls on
There's NOTHING on TV in Europe – American video DOMINATES
Even France's mega subsidies don't stop US content onslaught
UK Parliament rubber-stamps EMERGENCY data grab 'n' keep bill
Just 49 MPs oppose Drip's rushed timetable
Want to beat Verizon's slow Netflix? Get a VPN
Exec finds stream speed climbs when smuggled out
Samsung threatens to cut ties with supplier over child labour allegations
Vows to uphold 'zero tolerance' policy on underage workers
Dude, you're getting a Dell – with BITCOIN: IT giant slurps cryptocash
1. Buy PC with Bitcoin. 2. Mine more coins. 3. Goto step 1
prev story

Whitepapers

Top three mobile application threats
Prevent sensitive data leakage over insecure channels or stolen mobile devices.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.
Mobile application security vulnerability report
The alarming realities regarding the sheer number of applications vulnerable to attack, and the most common and easily addressable vulnerability errors.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.