Original URL: https://www.theregister.com/2014/03/20/sysadmin_security/
A sysadmin always comes prepared: Grasp those essential tools
Help us scope the perfect IT admin's toolkit
Posted in On-Prem, 20th March 2014 12:41 GMT
+Competition We have prizes up for grabs. The first 30 people to sign up for a GFI Cloud free trial and add two or more devices to the console get a limited edition T-shirt from The Register. All signs ups that add two or more devices before this competition ends will be entered into a draw to win a 10-user licence of GFI Cloud’s AntiMalware Pack. Find the full competition T&Cs here.
There are certain tools that all sysadmins need. Some, such as the venerable ping, are so fundamental that their lack would be considered an oddity.
Others, such as backups, should by rights be deployed absolutely everywhere, yet incomprehensibly are not. Debating which tools are best is the blood sport of our industry.
I remember the carefree days before broadband. I built open-air lasers to carry unencrypted traffic at a blistering 9600 bps.
Obscurity really was security. Only a handful of people in my area at the time could recognise what those rigs were designed to do, let alone intercept the communications. The chances were vanishingly small that of any of them would do so for malicious purposes.
Broadband arrived. With it came new threat vectors along with a technology industry exploding in innovation. Some of the biggest names in the industry were caught unawares – Windows XP famously had a firewall that wasn't on by default.
Windows XP's Service Pack 2 changed all this and finally made Microsoft's pivot towards "security first" tangible to the customer. It has done yeoman's work since then, but the industry as a whole cannot say the same.
The companies behind major culprits such as Java, Flash, PDF readers and consumer broadband routers haven't cleaned up their act despite years of continued assault.
In very human fashion, the companies behind the "Internet of Things" machine-to-machine revolution are proving no better. They are repeating the mistakes of their predecessors, offering poor support and units that are vulnerable by default.
The world has changed since my little 9600 bps lasers and threat models have changed with it. Have our sysadmin's toolkits kept up?
Let's take a peek at the major categories and have you, the reader, submit your thoughts on which tools are the best for the job.
Perimeter threat detection
This is about more than just standing up a firewall and hoping nobody crawls through. It is about assuming that someone eventually will and deploying tools to detect this when it happens.
Intrusion detection and prevention systems (IDPS) exist in any number of forms to attempt to detect the untoward activity.
Basic IDPS systems are designed to be deployed to monitor individual services or servers. Fail2Ban is a popular example.
Other IDPS systems are designed to scan active network streams and typically come in the form of application layer gateways (ALGs). Today, these are commodities, easily found as physical or virtual appliances.
Perhaps the most pervasive IDPS technologies deployed today are web filtering and email filtering. These can include everything from ad-blocking and anti-malware to spam filtering or blocking undesirable content.
They can be installed as part of a firewall/ALG appliance but are increasingly deployed on a per-system basis as part of a cloud service that offers rapid-release threat signatures.
Working to block threats at the perimeter is the first, and easiest, step towards a functional modern IT deployment. A large number of things that can go wrong simply don't if the bad guys can't get past the edge defences.
Extant threat detection
No battle plan survives contact with the enemy. No matter how sophisticated and well implemented your perimeter defences, something will inevitably get through.
In addition, you will have to cope with privileged users abusing their privilege, Pointy Haired Boss syndrome and inadequate funding.
Extant threat detection has to include various types of hardware, software and network monitoring. It needs to detect failed equipment, but also unbalanced configurations and runaway resource usage. It needs to be able to find configuration issues ranging from open ports to improper Group Policy Objects.
Event log monitoring is the easiest path forward. Operating systems and applications are usually pretty good about logging when something goes pear-shaped.
ACL auditing needs to be considered. This ranges from file permissions and network ACLs through to application-specific rights allocation. The trick is to have software that can keep an eye on all the logs across all systems and filter signal from noise.
Anti-malware software needs to be centrally managed, with regular updates and proper installation verified.
If something goes wrong admins need to be notified; the first sign of trouble many admins get that a system has been compromised is not detection by the anti-malware application, but rather the unceremonious murder of said application by malware that got in under the radar.
Somewhere in here we need to add USB scanning and efforts to uncover clandestine IT.
In some cases, these items are different tools but they are converging. Even where the individual components of extant threat detection aren't collapsing into a single tool, unified management of the various tools in this category is increasingly pervasive.
Entropy assurance
This is a relatively new category in mass public consciousness, but it is increasingly important.
Entropy assurance tools have one job: to generate high entropy to secure system and service access and manage all of it in a human-compatible fashion.
Here we find tools such as password managers and certificate and key management systems. In today's world of custom silicon, GPGPU computing and massive Amazon cracking setups, entropy assurance apps are no longer optional.
Desired state management
This category covers everything from the aged Group Policy Objects to patch management through to state enforcement technologies such as Puppet. Desired state management apps are simple in concept but miserable to design.
There are three elements to all desired state management tools: detection of current state; remediation (if current state does not equal desired state); and freaking out if remediation fails.
Desired state management tools are most popularly associated with the DevOps and automation movements but they have been around for decades in one form or another.
The number of widgets IT departments have to care for is exploding
Once optional, they are now vitally required tools for the simple reason that the number of widgets IT departments have to care for is exploding. Even the smallest of businesses cannot mollycoddle each and every device.
Of the desired state management tools, patch management is probably the most widely deployed. Virtually everyone has Windows Server Update Services (WSUS) but WSUS doesn't make sure Java is up to date, nor keep Flash under control.
Windows is not the point of vulnerability it once was; it is the third party apps – on Linux and Apple as well as Windows – that are often the problem. This means you need to have patch management that can cover more than WSUS alone can handle.
Asset management
Asset management applications have reached must-have status. Today's world consists not merely of PCs and switches but of smartphones, NFC readers and cloud services.
IT departments have to track hardware, software and service subscriptions. They need to know who has what and why, and whether the ongoing expenses are still justified.
More critically, asset management tools are increasingly vital to planning backups and various tiers of network design, and detecting clandestine IT deployments.
Anything that isn't in the system is a potential security threat, and with the Internet of Things that is only going to get worse.
Backups and disaster recovery
Not so long ago I had a conversation with Jamie Brenzel, CEO of Canadian cloud backup provider KineticD. I heard the spiel on what the company does and noted that by and large its offerings were pretty bare.
Backup is a crowded space and I wondered how it managed to survive the technological arms race.
Brenzel's answer was simple, if shatteringly depressing: an alarming number of businesses simply don't have backups of any kind. KineticD is not competing against other backup vendors, it is competing against "nothing at all."
It should go without saying that if you have no other tools to hand you at least have working backups. Sadly and incomprehensibly, this is not universally the case.
Proper disaster recovery planning is even rarer. Despite this, our businesses increasingly depend upon complex automated IT to function.
We manage to stay competitive only because we have replaced expensive, salaried people with cheaper (in theory) machines.
Retraining people is a huge pain and productivity drain, but rebuilding data and redesigning automation causes similar grief.
Diagnostics
Just as perimeter defences aren't enough for any business, all of the other categories discussed here combined will not prevent all failures. Eventually, something somewhere is going to go mad and you will need the tools to figure out what has gone wrong.
Basic tools such as ping are so common as to be known even to much of the non-IT populace. Others like packet sniffers are useful for finding out which widget is chanting bing tiddle tiddle bong instead of doing its job.
For every potential error there are a dozen individual tools – I can mention at least eight packet sniffers off the top of my head – but as the number of basic elements sysadmins have to diagnose continues to increase, tool suites are becoming increasingly common.
These suites range from specially designed Linux Live CDs to quasi-legal DVDs of commercial software such as Hirens and installable omnitools such as LanGuard all the way to cloudy subscription diagnostics-as-a-service offerings.
As-a-service management
This emerging category can range from outage notification to root-cause identification. The real purpose of these tools is blame assignation.
A cloud provider doesn't have a neck to wring, so when something goes pear shaped the first neck to reach for is that of the nearest sysadmin.
Tools are emerging to fill this niche. First to mind is startup Thousand Eyes, which does a good job of telling you whether the problem is yours, the service provider's or that of one of the ISPs between you.
Developing orthogonal paths to problem resolution (read finding out who to send the bottle of Scotch to) is also a critical tool in the as-a-service era.
Have your say
What started all of this for me was an introduction to GFI Cloud. This is a tool that offers many of the above must-have features with an ultra-simple interface.
GFI Cloud is quite a bit different from the company’s on-premises offerings and it got me thinking about the evolution of the systems administrator’s professional toolkit.
The categories outlined above are pretty broad but I think they cover the critical areas. Which tools do you feel are best in each category? Which vendors do you feel have proved themselves? More importantly, who makes tools that cover multiple categories in a single offering?
If I have missed a category, please mention it along with the tools you feel solve the problems listed. Let us know what your biggest sysadmin headaches are and your favourite tools to fix them.
Win a prize
We have prizes up for grabs. The first 30 people to sign up for a GFI Cloud free trial and add two or more devices to the console get a limited edition T-shirt from The Register.
All signs ups that add two or more devices before this competition ends will be entered into a draw to win a 10-user licence of GFI Cloud’s AntiMalware Pack.
Find the full competition T&Cs here. ®