Application QoS – where's the measuring tape?
Taking the business view of metrics
IT has undergone a major development in the last few years, namely the acknowledgment that end users have little or no interest in how the application or service is delivered to them - so long as it is delivered and meets their own service level criteria.
The background mechanics of the delivery should be of no concern to the business. In other words, IT is almost at a stage when, to a large degree, users can regard the service almost as a utility: turn on, use, turn off. More pertinently, users now expect IT services to be available whenever they wish, wherever they happen to be. This throws into the spotlight the question of how best to measure the “quality of IT service delivery”
Today, “application management” consists mostly of monitoring the functions of major business applications and the supporting IT infrastructure. But this IT view of applications tends not to take into account the metrics that most matter to business users, namely some indication of business oriented performance, such as end user application response time or some other metric that illustrates, in terms understandable by a line-of-business user, the level of service delivery.
What kinds of metrics would make more sense from a business perspective? The numbers of orders that may be processed per hour, the times during which service will be delivered and supported and the end users response times for particular transactions are examples. Service Level Management tools are now reaching a maturity that is bringing this intriguing area to the forefront in many organisations, a development that has been sadly neglected until recent times.
As the figure above illustrates, the use of SLA management using business metrics is a focus of attention in very many organisations, a result highlighted in a Freeform Dynamics report.
This need to measure real application response times and availability from the perspective of a user has created a demand for tools that address end to end application monitoring. A number of vendors supply such tools that provide these measurement capabilities using either agent-based topologies or “robots” that simulate end users transactions. Each method has its advantages and disadvantages.
Agent-based systems deploy small programs on the systems of end users to measure their use of the applications and measures the end to end response times of the transactions. The advantage of this method is that real transactions are measured on real user systems. The disadvantage is that the tool depends upon the user being at work and accessing the services to be measured. To avoid receiving no information it may be required that many agents be deployed and this can result in an avalanche of data being collected.
Robot transaction simulation systems can be programmed to perform regardless of user activity and can provide good results even when user activity is low - or performance is so slow that real users abandon the service. But few organisations are prepared to allow robot simulations to update their real data systems. Consequently, robots may not provide a complete picture of the service delivered to real users. It appears that with the tools available today, a combination of agent based and transaction simulation systems would provide the best solution to the issue of measuring service level delivery.
The end-to-end monitoring market is developing apace and sophisticated functionality is now available to meet the growing demand for service level monitoring and reporting. There is much room for these systems to integrate with enterprise management tools to provide valuable additional information to global monitoring systems and root cause analysis of infrastructure issues.
Couple this with virtualisation solutions, and effective change management processes, and IT staff have the key to service delivery in their hands and also the potential to both maximise the benefit that IT can deliver to the business but also to show that they are doing so.
Nonetheless, these tools must be used with discretion and it is essential that some basic questions are asked before SLA monitoring takes place. Chief among these are the absolute requirement to identify what services are important to users and to ensure that reasonable, measurable SLAs are agreed. With this knowledge, appropriate technology tools can be put in place to supply the measurements that qualify the quality of the business service that is delivered, rather than some arbitrary technical criteria.