Feeds

How spreadsheets (nearly) conquered and killed the financial industry

Would you trust millions on an Excel formula?

Secure remote control for conventional and virtual desktops

Step forward the spreadsheet, saviour of those chained to change requests

Though there seemed to be no hope of having this process simplified, there was a tacit recognition by management that it was completely unworkable in the dynamic and fast-moving world of spreadsheet development by “rapid deployment” teams working closely with the business. (I managed one of these teams, with members in London, New York, and Hong Kong, each supporting their local desk.)

For these developers, unlike the teams who supported larger-scale, Linux-and-Java-based systems, the usual rules simply didn’t apply. Spreadsheet changes could be pushed out at will, and at a moment’s notice.

In addition to removing a tremendous amount of drudgery and bureaucracy from these developers’ lives, this across-the-board exemption made it much easier for traders to get their day-to-day work done, since their requests could be implemented almost immediately.

This resulted in a kind of positive feedback loop, where desk developers began to be seen as the only ones who could deliver usable systems when they were needed.

But while it was true that development of large systems was tortuously slow and horribly error-prone, it was clear to anyone working in technology that there were many reasons for this divergence: in addition to being burdened with the tedious rollout procedure described above, teams tasked with building industrial-strength systems suffered from top-heavy management, uncooperative system administrators and support staff (one of many symptoms of an out-of-control bureaucracy), and a distance from the desk that proved disastrous — all in sharp contrast to the rapid deployment teams, who sat next to the traders all day and are were intimately familiar with their workflows.

Furthermore, the design and coding of a Java system is always significantly more involved than relatively simple Excel scripting, making it easy for people doing the latter to become instant heroes.

Excel with Excel

All these factors led to an ever-increasing reliance on rapid-deployment teams and spreadsheets. Large development projects either failed outright or couldn’t be launched to begin with because they needed big teams and long-term horizons for the design, development, and QA.

So the desk frequently went with the “stopgap” of a spreadsheet.

This would invariably be seen as tactical (i.e., short-term), but again and again, when the more permanent system never arrived, the spreadsheet became the de facto strategic solution, much easier than trying to address the bigger problems that were causing so many large-scale development projects to falter. As a result, many processes that were plainly too big or complex to run in Excel over the long term did.

The result: frequent memory blowouts, painfully slow calculation cycles, and PCs pushed past way their limit. With piles of money riding on the deals that were being priced, or the risks that were being hedged, these seemingly minor PC hiccups had the potential to cause not-at-all-minor losses.

But the temptation to go with the quick solution was too great, and large-scale development too trouble-plagued, for most people to look at this state of affairs objectively.

While the example I’ve just described was the most extreme I’ve seen, the tendency to reach for the quick and dirty solution of an Excel spreadsheet whether appropriate or not is common on trading desks everywhere. Traders like the idea of having direct control of their processes, or at least being able to see their workings; large-scale systems become harder to design and build as they get more complex, and the development teams harder to manage; and rapid-deployment teams are generally free from the burdensome rules that other developers have to adhere to, making them more agile in their response to fast-changing business requirements.

But this short-sightedness can lead to a variety of problems: relying on lightweight, desktop-based solutions for business-critical functions; giving too much power to users in situations that require centrally managed checks and balances; and circumventing the professional monitoring and full-time production support that naturally come with large-scale systems.

Giving users too much control and flexibility can be a very bad idea when the processes involved are especially delicate, or where the amount of money at stake is large enough to render even an innocent mistake disastrous. ®

Secure remote control for conventional and virtual desktops

More from The Register

next story
Ex-US Navy fighter pilot MIT prof: Drones beat humans - I should know
'Missy' Cummings on UAVs, smartcars and dying from boredom
Facebook, Apple: LADIES! Why not FREEZE your EGGS? It's on the company!
No biological clockwatching when you work in Silicon Valley
The 'fun-nification' of computer education – good idea?
Compulsory code schools, luvvies love it, but what about Maths and Physics?
Doctor Who's Flatline: Cool monsters, yes, but utterly limp subplots
We know what the Doctor does, stop going on about it already
Happiness economics is bollocks. Oh, UK.gov just adopted it? Er ...
Opportunity doesn't knock; it costs us instead
'Cowardly, venomous trolls' threatened with TWO-YEAR sentences for menacing posts
UK government: 'Taking a stand against a baying cyber-mob'
Sysadmin with EBOLA? Gartner's issued advice to debug your biz
Start hoarding cleaning supplies, analyst firm says, and assume your team will scatter
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.