Tune your service oriented applications with SQL
Bottleneck, meet database
Performance is one of the more insidious problems that a developer can face. We don't seem to have time for it with all work just getting an application's functionally correct.
The problem is that it's a non-functional requirement and as with all non-functional requirements the specification is often either vague or non-existent. It can be difficult to test performance until late in the development cycle and - trust me when I say this - late in the end of the development cycle is not a good place to discover performance problems.
With Oracle fresh in our minds thanks to OpenWorld, I want to look at the performance problem with regard to the database. Profiling and algorithmic complexity are - of course - important, but as the world becomes more distributed and service oriented we should consider if our performance dollars can be better spent improving our SQL queries. Does this sound crazy?
According to author and industry expert Gavin Powell, an order of magnitude application performance increases can be obtained simply by tuning the SQL and correctly optimizing the data access path.
This is an incredible figure and it begs the question: can developers afford to continue leaving SQL expertise to the DBA? After all, if such massive gains are regularly possible then doesn't this mean that a lot of developers are producing pretty questionable SQL? We can't argue that it's not important because it's definitely not cheap to scale the database that's for sure.
What's making it worse is that in our connected world performance bottlenecks are more likely to be discovered. In the past, barriers to integration such as obscure APIs and thick clients kept information within the application and only the most stubborn external aggregators could access it.
These days, web services and service oriented architectures (SOAs) mean we are no longer protected from scalability and performance issues. Increasing aggregation of information is pushing demands on software service infrastructure and any bottlenecks in the system will be exposed sooner or later. If the system grinds to a halt then the chances are it could spell project failure - and that does not look good on the CV!
We need to accept that application performance needs to be explicitly managed. It's good that we implement use cases and write tests that demonstrate functional correctness. Unfortunately, functional tests that use minimally populated databases do not tell us anything about the performance characteristics of a system under real world data volumes with parallel processes making simultaneous demands on the database.