Original URL: http://www.theregister.co.uk/2009/11/24/ask_the_expert_dynamic_management_workload_balancing_skills/

Data centres: what are the new skill sets?

Commodity technology = commodity staff?

By Team Register

Posted in Server Management, 24th November 2009 10:03 GMT

You the expert The capabilities of modern servers offer in-principle benefits of more dynamic management, workload balancing and so on. How do these capabilities impact on the skillsets required of data centre operations staff today?

Trevor PottTrevor Pott
Infrastructure Support Engineer

The commoditisation of certain technologies once found only in high-end servers has changed the face of IT. Once upon a time only the most high end systems had the kinds of remote management, live workload transfer, high availability and "sandboxing" that all but the smallest IT shops have started to take for granted.

Not long ago the ability to remote-manage a commodity server outside the OS was a pipe dream. Today the purchase of hardware without such considerations is almost inconceivable. IPKVMs and Lights Out Management (LOM) solutions are increasingly affordable and becoming a standard component in all business class systems. (Think VPro and AMT.)

Virtualization management suites can give you the ability to turn your collection of x86 and x64 servers into something that behaves very much like one big computer. You can fill this "meta computer" with VMs, each of which can serve as a sandbox for a given workload. VMs can be moved from node to node on-the-fly. Neat tricks like mass clone deployment or high-availability are now commonplace.

While all of the above has many consequences, the take-home message here is that the technology available today is enabling any given number of administrators to care for a greater number of servers than ever before. For all intents and purposes the traditional "grunt work" of IT has been largely automated away.

In large organisations this often means IT administration services become concentrated in a central location which no longer has to be physically near a datacenter. Small numbers of "rack monkeys" are left to man the datacenters and swap out the dead bits. As new technologies allow IT departments to become leaner, the altered administrator-to-server ratio merely increases the burden of responsibility placed on each administrator.

For the moment, smaller organisations still cling to "jack of all trades" administrators. Though features like LOM and virtualisation are increasingly common, the management tools to really make them shine are often outside the reach of an SME budget.

A management tool licensing stack costing half again as much as the hardware of your 2-socket server is still finding a hard sell here. Costs of these tools are however dropping, and effective remote management of SME IT services by third-parties is increasingly a viable consideration.

These technologies are inducing a massive change in the demand for skills among administrators. As smaller numbers of specialist administrators are increasingly able to manage larger numbers of more complex systems, the "jack of all trades" administrators have become an endangered species. The commoditisation of server management tools has thus begun a true commoditisation of the various skill sets of administrators.

Adam SalisburyAdam Salisbury
Systems Administrator

The pace at which technology evolves has always led staff across the whole of the IT industry in a perpetual chase for skills, knowledge and experience and just as you've mastered the latest industry development the 'next big thing' is already looming large on the horizon. One technology, largely disregarded until recently is power management technology. The race for the highest performance figures is fast becoming a race for the most efficient performance figures.

While the steady refinement of individual power management puzzle pieces does continue in the form of more efficient hardware and better firmware, there are now entire suites of software being developed to manage the power of entire estates, from out-of-band embedded software reporting on individual server power consumption to advanced IP-aware dynamically adjusted UPS systems able to communicate with one another at branch office level to make better use of the juice. Gone are the days where power management simply meant duplicated mains supplies, generators and batteries.

Even the big server virtualisation players have developed systems which can dynamically shift workloads between hypervisors during periods of low demand and powering down the servers not currently in use, for the small players the onus is normally on one or two system administrators to learn, deploy and manage these technologies whereas for some enterprises such a technology deployed a large scale environment will result in entirely new teams of operations staff being formed to manage what is fast becoming another 'dimension' of IT.

In terms of the impact to staff of these fledging technologies is that like virtualisation for instance, the level of knowledge and skills currently required to make effective use of the technology is fairly low in comparison with other areas. Converting from Active Directory to eDirectory for instance, would present a far bigger issue in terms of knowledge or skills gap than picking up a brand new technology. That said, with all that is currently expected of an average support technician in an average support role it's still a big challenge.

In many cases assessing the impact of new technological requirements upon those who must support them is inherently difficult, not least because knowledge and skills can be somewhat subjective to measure. The majority of a technical workforce is well prepared and equipped to self-learn to an extent, requesting further formal training only when they've achieved all they can on their own.

Currently the vendors of these advanced power management systems do offer training on such products, but more mainstream training is still in the making: no doubt in the future as such technology becomes commonplace and essential rather than merely advantageous as it is now, a world of support and learning will rise up around it.

For now at least, the best way to prevent overwhelming the operations staff is to research, learn and adopt such technology early and thus close as far as is possible, any gap that may exist within the organisation. Be ready to capitalise on the technology when the tools to wield it do become available.

Tony LockTony Lock
Programme Director, Freeform Dynamics

It is correct that many new server and, especially, server virtualisation systems offer the ability to manage systems far more dynamically than ever before. Indeed, it is fair to say that until now the primary mandate adopted by most IT organisations has been "if it is not broken, leave it alone". The capabilities inherent in many systems to move workloads around very quickly or to create and take down virtual servers and their associated workloads in a matter of minutes open the door to new operational modes.

At the heart of the matter is the question of server resource allocation and how to obtain "optimal" benefits from the systems at hand. From a skills perspective there is still the need to be able to monitor and manage the complex systems at all levels of their operations. But the very idea of being able to alter the configurations of systems as a routine part of standard operations will eventually demand that new skills, or at the least operational procedures to be put in place.

Chief amongst these will be some way of selecting what proportion of available resources is allocated to each application in the dynamic infrastructure. Clearly if the infrastructure has more resources than can be consumed under any likely workload scenario, there will not be too many demand clashes to handle.

But if - as is likely - organisations seek to limit the overall size, and hence cost, of the IT infrastructure, there will be times when someone will need to decide how constrained resources are allocated. This will require good "vision" of the likely business consequences of such choices.

Thus the IT department will need to possess the skills and monitoring tools to make such judgments, or put some form of automated policy prioritisation processes in place. Getting hold of either of these will require no small measure of communication and diplomatic skills as well as a good handle on business reporting systems.

Ian BeckinghamIain Beckingham
Manager of the Enterprise Technical Specialist team in EMEA, for Intel.

With new servers and technologies such as virtualisation, data centre skillsets are evolving. The new frontier of cloud is pushing IT departments and Data Centre Staff people to move their skill set from traditional single server, single OS, single application to a new level.

The sharing of resources like computing, networking and storage across different OSes and application have huge impact. Managing such a Dynamic IT Infrastructure requires a new global approach and awareness to the Data Center.

Think back several years - if a user wanted to measure performance on a database, they just needed to lookout the number of transactions per second. To find out a CPU bottleneck it was just enough to have a look at the CPU load.

Nowadays, that same database may be virtualised on a server with virtual networking and virtual storage. Identifying the bottleneck(s) becomes harder.

Although the skillsets are changing, the skillsets of IT staff need to converge as the resources do. People dealing just with servers and OSes in the past need now to know about networking and storage; network administrators will not be able to ignore the storage and the virtual computing environments, and the storage administrators will also need to open to networking and to Computing to fully understand the needs of the modern computer infrastructures.

Intel is helping to make things more transparent and less complicated through technology features in the platform eg Intel Virtualisation Technology. However, this also means that the skillsets of IT staff will need to evolve.