Feeds

How spreadsheets (nearly) conquered and killed the financial industry

Would you trust millions on an Excel formula?

Internet Security Threat Report 2014

Step forward the spreadsheet, saviour of those chained to change requests

Though there seemed to be no hope of having this process simplified, there was a tacit recognition by management that it was completely unworkable in the dynamic and fast-moving world of spreadsheet development by “rapid deployment” teams working closely with the business. (I managed one of these teams, with members in London, New York, and Hong Kong, each supporting their local desk.)

For these developers, unlike the teams who supported larger-scale, Linux-and-Java-based systems, the usual rules simply didn’t apply. Spreadsheet changes could be pushed out at will, and at a moment’s notice.

In addition to removing a tremendous amount of drudgery and bureaucracy from these developers’ lives, this across-the-board exemption made it much easier for traders to get their day-to-day work done, since their requests could be implemented almost immediately.

This resulted in a kind of positive feedback loop, where desk developers began to be seen as the only ones who could deliver usable systems when they were needed.

But while it was true that development of large systems was tortuously slow and horribly error-prone, it was clear to anyone working in technology that there were many reasons for this divergence: in addition to being burdened with the tedious rollout procedure described above, teams tasked with building industrial-strength systems suffered from top-heavy management, uncooperative system administrators and support staff (one of many symptoms of an out-of-control bureaucracy), and a distance from the desk that proved disastrous — all in sharp contrast to the rapid deployment teams, who sat next to the traders all day and are were intimately familiar with their workflows.

Furthermore, the design and coding of a Java system is always significantly more involved than relatively simple Excel scripting, making it easy for people doing the latter to become instant heroes.

Excel with Excel

All these factors led to an ever-increasing reliance on rapid-deployment teams and spreadsheets. Large development projects either failed outright or couldn’t be launched to begin with because they needed big teams and long-term horizons for the design, development, and QA.

So the desk frequently went with the “stopgap” of a spreadsheet.

This would invariably be seen as tactical (i.e., short-term), but again and again, when the more permanent system never arrived, the spreadsheet became the de facto strategic solution, much easier than trying to address the bigger problems that were causing so many large-scale development projects to falter. As a result, many processes that were plainly too big or complex to run in Excel over the long term did.

The result: frequent memory blowouts, painfully slow calculation cycles, and PCs pushed past way their limit. With piles of money riding on the deals that were being priced, or the risks that were being hedged, these seemingly minor PC hiccups had the potential to cause not-at-all-minor losses.

But the temptation to go with the quick solution was too great, and large-scale development too trouble-plagued, for most people to look at this state of affairs objectively.

While the example I’ve just described was the most extreme I’ve seen, the tendency to reach for the quick and dirty solution of an Excel spreadsheet whether appropriate or not is common on trading desks everywhere. Traders like the idea of having direct control of their processes, or at least being able to see their workings; large-scale systems become harder to design and build as they get more complex, and the development teams harder to manage; and rapid-deployment teams are generally free from the burdensome rules that other developers have to adhere to, making them more agile in their response to fast-changing business requirements.

But this short-sightedness can lead to a variety of problems: relying on lightweight, desktop-based solutions for business-critical functions; giving too much power to users in situations that require centrally managed checks and balances; and circumventing the professional monitoring and full-time production support that naturally come with large-scale systems.

Giving users too much control and flexibility can be a very bad idea when the processes involved are especially delicate, or where the amount of money at stake is large enough to render even an innocent mistake disastrous. ®

Providing a secure and efficient Helpdesk

More from The Register

next story
Facebook, Apple: LADIES! Why not FREEZE your EGGS? It's on the company!
No biological clockwatching when you work in Silicon Valley
'Cowardly, venomous trolls' threatened with TWO-YEAR sentences for menacing posts
UK government: 'Taking a stand against a baying cyber-mob'
Doctor Who's Flatline: Cool monsters, yes, but utterly limp subplots
We know what the Doctor does, stop going on about it already
Happiness economics is bollocks. Oh, UK.gov just adopted it? Er ...
Opportunity doesn't knock; it costs us instead
Arab States make play for greater government control of the internet
Nerds told to get lost in last-minute power grab bid at UN meeting
Zippy one-liners, broken promises: Doctor Who on the Orient Express
Series finally hits stride, but Clara's U-turn is baffling
Don't bother telling people if you lose their data, say Euro bods
You read that right – with the proviso that it's encrypted
Apple SILENCES Bose, YANKS headphones from stores
The, er, Beats go on after noise-cancelling spat
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.