Feeds

How spreadsheets (nearly) conquered and killed the financial industry

Would you trust millions on an Excel formula?

Maximizing your infrastructure through virtualization

Step forward the spreadsheet, saviour of those chained to change requests

Though there seemed to be no hope of having this process simplified, there was a tacit recognition by management that it was completely unworkable in the dynamic and fast-moving world of spreadsheet development by “rapid deployment” teams working closely with the business. (I managed one of these teams, with members in London, New York, and Hong Kong, each supporting their local desk.)

For these developers, unlike the teams who supported larger-scale, Linux-and-Java-based systems, the usual rules simply didn’t apply. Spreadsheet changes could be pushed out at will, and at a moment’s notice.

In addition to removing a tremendous amount of drudgery and bureaucracy from these developers’ lives, this across-the-board exemption made it much easier for traders to get their day-to-day work done, since their requests could be implemented almost immediately.

This resulted in a kind of positive feedback loop, where desk developers began to be seen as the only ones who could deliver usable systems when they were needed.

But while it was true that development of large systems was tortuously slow and horribly error-prone, it was clear to anyone working in technology that there were many reasons for this divergence: in addition to being burdened with the tedious rollout procedure described above, teams tasked with building industrial-strength systems suffered from top-heavy management, uncooperative system administrators and support staff (one of many symptoms of an out-of-control bureaucracy), and a distance from the desk that proved disastrous — all in sharp contrast to the rapid deployment teams, who sat next to the traders all day and are were intimately familiar with their workflows.

Furthermore, the design and coding of a Java system is always significantly more involved than relatively simple Excel scripting, making it easy for people doing the latter to become instant heroes.

Excel with Excel

All these factors led to an ever-increasing reliance on rapid-deployment teams and spreadsheets. Large development projects either failed outright or couldn’t be launched to begin with because they needed big teams and long-term horizons for the design, development, and QA.

So the desk frequently went with the “stopgap” of a spreadsheet.

This would invariably be seen as tactical (i.e., short-term), but again and again, when the more permanent system never arrived, the spreadsheet became the de facto strategic solution, much easier than trying to address the bigger problems that were causing so many large-scale development projects to falter. As a result, many processes that were plainly too big or complex to run in Excel over the long term did.

The result: frequent memory blowouts, painfully slow calculation cycles, and PCs pushed past way their limit. With piles of money riding on the deals that were being priced, or the risks that were being hedged, these seemingly minor PC hiccups had the potential to cause not-at-all-minor losses.

But the temptation to go with the quick solution was too great, and large-scale development too trouble-plagued, for most people to look at this state of affairs objectively.

While the example I’ve just described was the most extreme I’ve seen, the tendency to reach for the quick and dirty solution of an Excel spreadsheet whether appropriate or not is common on trading desks everywhere. Traders like the idea of having direct control of their processes, or at least being able to see their workings; large-scale systems become harder to design and build as they get more complex, and the development teams harder to manage; and rapid-deployment teams are generally free from the burdensome rules that other developers have to adhere to, making them more agile in their response to fast-changing business requirements.

But this short-sightedness can lead to a variety of problems: relying on lightweight, desktop-based solutions for business-critical functions; giving too much power to users in situations that require centrally managed checks and balances; and circumventing the professional monitoring and full-time production support that naturally come with large-scale systems.

Giving users too much control and flexibility can be a very bad idea when the processes involved are especially delicate, or where the amount of money at stake is large enough to render even an innocent mistake disastrous. ®

The Power of One Infographic

More from The Register

next story
BBC goes offline in MASSIVE COCKUP: Stephen Fry partly muzzled
Auntie tight-lipped as major outage rolls on
There's NOTHING on TV in Europe – American video DOMINATES
Even France's mega subsidies don't stop US content onslaught
You! Pirate! Stop pirating, or we shall admonish you politely. Repeatedly, if necessary
And we shall go about telling people you smell. No, not really
Airbus promises Wi-Fi – yay – and 3D movies (meh) in new A330
If the person in front reclines their seat, this could get interesting
UK Parliament rubber-stamps EMERGENCY data grab 'n' keep bill
Just 49 MPs oppose Drip's rushed timetable
Want to beat Verizon's slow Netflix? Get a VPN
Exec finds stream speed climbs when smuggled out
Samsung threatens to cut ties with supplier over child labour allegations
Vows to uphold 'zero tolerance' policy on underage workers
Dude, you're getting a Dell – with BITCOIN: IT giant slurps cryptocash
1. Buy PC with Bitcoin. 2. Mine more coins. 3. Goto step 1
prev story

Whitepapers

Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.