Borland throws down Gauntlet to software defects
Taking the gloves off
Software is unstable, full of bugs and bloated. Developers spend so much time adding new features that software testing is a last minute thing. Even the mantra of "designed secure" has failed to really improve the quality of software - just witness the number of day zero attacks against Microsoft.
Enter Borland, which believes that it has the answer to your problems; if you're a software developer, that is. Gauntlet (there’s a Borland Quickstart for Gauntlet here) is a product that Borland acquired last year and has quickly progressed to a production release. This does mean that it has a "first version" feel at times but this doesn't detract from the core message. That message is: “test now, test often, test always”.
It's a good message; in fact, it's a great message. In a nutshell, Gauntlet achieves this by running tests every time a piece of software is checked in to the software repository. In fact, it doesn't have to be just when your piece of software is checked in; it could be when anything you depend on is changed and checked in. Then the tests can run against an entire software manifest. Too good to be true? Well, no! That isn't to say they've completely cracked it, but they have done a very good job in this first release and it deserves a serious look.
Installation is simple but there are a few requirements which are not all checked. You do need a valid Java Virtual Machine (JVM) and Java Development Kit (JDK). Like most software vendors, Borland chooses not to distribute these with its software; instead, it tells you to download them before installing it. You also need to have a Subversion source code configuration client installed on the computer. This is not provided and unless you check the documentation carefully, you could be forgiven for missing it. Quite why Borland opted to use Subversion, rather than shipping a copy of its own StarTeam product, is a mystery. I know it wants to be Eclipse-centric these days - but this shouldn't come at the cost of ignoring its own products.
Once installed, you need to do a little configuration. You start by creating repositories and sandboxes.
These are important, as each developer works in his/her own sandboxes. This prevents any problems from their code affecting other members of the development team. As a project moves closer to completion and you have developers collaborating more closely on the code, you can have developers sharing a sandbox. Conversely, in the early days, a developer can have access to multiple sandboxes. Users are defined by their roles (groups) and you can either add users directly into Gauntlet’s own authentication system; or integrate it with other solutions such as Active Directory. This was very easy to do.
The workspace looks quite cluttered. It's a tabbed interface; well sort of. Instead of tabs, you have a series of icons across the top of the workspace and each takes you to its own area. Down the left, you have a menu system, which lets you get to some commands more quickly. Creating a project and its associated sandboxes is very simple but you still need to go to the command prompt and add in some directories. See what I mean about being a “first version” product? Hopefully, by the time the next version arrives, Borland will have dealt with this.
Sandboxes are critical to Gauntlet’s behaviour. Developers check out code and work in a sandbox. When they have finished they check in code. At this point, JUnit tests, otherwise known as “Gauntlets” are executed. These run checks against the code. When you write a Gauntlet, you define an XML file which determines which tests are executed at check-in. If you want to ensure that an entire dependency tree is tested, then you say so in the Gauntlet XML file. The build is actually done using ANT, which looks for either gauntlet-adapter.xml or build.xml. For this to work there must be one or the other in the build directory. What is seriously missing is the ability to manage and version the tests themselves. There is no process whereby tests can be allocated to a range of different pieces of code. And there’s no control mechanism to show where a test is or is not being used. This means that you cannot find all instances of a test unless you walk the directory tree or write a script to do it. This must be something that Borland addresses - if this tool is to get the wider audience it deserves.
The Gauntlets do more than simply look for failure conditions; they can be programmed to deal with code coverage. This is very cool. Software bloat is a nightmare. A significant amount of code in a lot of software is old and has been there since the first version. It never gets removed because no-one is sure what it does and how it is used. Using code coverage you can see what code is called and how often. Conversely, you can see what isn't called at all, which lets you slim down software with less risk. Only when a piece of code passes the Gauntlets can it be promoted into the main build tree. This should reduce the number of common and silly errors in software. It should also drastically reduce the time taken to track down faulty code as software projects near completion as the problems will already have been weeded out.
Borland does need to do a number of things to make this better. Address the issues already raised here is the first. Add a workflow so that I can see the history of a piece of code and use project management over the software should be a second one. This way each piece of code can be marked up as a separate task and passing the Gauntlets will enable it to progress through the project plan.
Overall, however, Borland has done a good job in the comparatively short time it has spent on Gauntlet so far - but there is much more to be done. Compared to some other Borland products over the last few years, nevertheless, Gauntlet is well ahead of the curve in terms of completeness and usable features. It just needs finishing off!®