Electric Cloud creating a Build storm

Reg Dev: OK, now at the nuts and bolts level, how do you actually make Build more efficient?

We tackle the build and release process at three levels – we automate it, accelerate it, and analyse it. Looking at each in turn:

• Automation: Most organisations have a homegrown solution for invoking and managing build and release tasks, comprised of scripts and servers and perhaps open source tools. The problems with a manual approach are numerous: they typically require manual intervention for ongoing maintenance, they don't scale to support multiple projects or teams, and they're inherently brittle. We've developed the ElectricCommander tool to address these issues at an enterprise level. ElectricCommander distributes individual jobs in parallel across multiple servers for faster throughput and efficient resource utilisation. Tasks can be run on demand, on a defined schedule, or continuously (i.e. a build is invoked whenever something is checked into a specified branch). It's based on a three tier architecture that can scale to manage any size project and any volume of tasks. It will also help teams respond nimbly to change, requiring minimal script changes to get started or make changes to the system on the fly. It also provides the reporting that manual approaches typically lack, to provide visibility into key trends and status of build and release tasks.
• Acceleration: This is where we hold several patents. The idea of distributing a build across multiple machines is deceptively simple. In reality, there are thousands of dependencies in a build of any reasonable size, so when you run processes in parallel a missing or undetected dependency will cause the build to fail or recompile in the wrong order. Our patented conflict detection and correction technology determines exactly which files were used to build every object file, library or executable, such that when build steps are run out of order, it automatically re-runs them in the correct order. So ElectricAccelerator produces a correct build, even when scaling to dozens of nodes in the cluster. With this technology, we typically see 8-12x speed improvements, with some customers realising as much as a 20x improvement.
• Analysis: In most organisations builds are black boxes. Unless it's a brand new project, no one is terribly sure what's happening in the build and they probably don't want to. In many organisations, being the "build master" is a punishment. We've shone a light on the build in ways that empower both developers and managers to really understand what's happening and take action to optimise, repair, or fine-tune the build. Our ElectricInsight tool mines the output of ElectricAccelerator to create a graphical representation of the build structure for performance analysis. You can literally, with a click, see for each job which files were used, and where dependencies may lie. Without this, doing this type of analysis would require pouring over thousands of pages of log files. At a higher level, our ElectricCommander tool features reports covering individual projects as well as cross-project views.

Reg Dev: I notice you mention 'several patents'. As an aside, this is a hot topic amongst developers. What protection does this really give you? Surely, a big company could break your patents and enmesh you with the slowest patent lawyers in the world – by the time you'd won, you'd have lost the business? And, aren't granted patents in the public domain where they can inspire genuine 'reverse engineers'?

Patterson: Patents are certainly a topic of some controversy. Many engineers worry that patents may impede the adoption of new technologies, particularly in fast-moving areas such as software. Others wonder how much protection patents really provide, and argue that the greatest barrier to entry for competitors is the amount of functionality in our products: it would take a competitor a long time to re-create what we've done, even without patent issues, and by then we will have implemented a lot of additional functionality. But the investment community believes that patents add value to the company, and we think they will also help us to defend against the most flagrant abuses of our intellectual property assets. Finally, they give us negotiating chips if other companies come to us with patent claims. The patent system definitely isn't perfect, but we still think it has value.

Reg Dev: Finally, does all this mean that Build/Release is now a finished technology? Or is there scope for improved features and even more efficiency? If so, please describe...

Patterson: ElectricAccelerator (for accelerating builds by distributing them across a cluster of machines) is the more mature technology, but we're always looking for ways to leverage this expertise. We've built significant knowledge around distributed processes and cluster computing – what are other processes we can accelerate or streamline using a distributed approach? We'll see.

On the other hand, the ElectricCommander product is an inherently extensible tool. At its core, it is a process automation framework. Build is one process among many. We've already developed integrations with SCM tools, testing tools, and are looking at the next wave of integrations with defect tracking tools and solutions at the deployment end. The more the build stage is tied into the overall development lifecycle at a process level, the more overall efficiencies developers will realise. That's been a challenge historically – ALM has traditionally been comprised of a series of disconnected tools and people. Now that vendors such as ourselves are bringing automation to the build and release cycle, we in the larger development tool industry have the technology and process knowledge to deliver on the promise of integrated ALM now. ®