Adding trendy tech SIEM to a hybrid computing setup
When a public cloud provider's involved, things get tricky
As I write this, Security Information and Event Monitoring is considered rather hip and cool. Everyone's talking about it, and the vendors of SIEM software are promoting the life out of it.
The thought process that prompts consideration of SIEM is: “No matter what I do to protect myself, an attack is possible so I need to pre-empt the need for diagnosis and forensics”.
SIEM is great for proactive, scheduled reporting – it allows you to spot things of interest and potential issues before anyone actually exploits them – but it comes into its own when you need to weed through the logs to find out what happened and how.
What does it do?
In its basic form a SIEM system goes against the principles you've designed into your infrastructure. Because what you do is configure all your event logs so they are sent (or at least a copy of them is sent) to the central repository that is the SIEM system. We'll come back to why that's not necessarily a bad thing shortly. The SIEM system then stores the data, collates it, reports significant findings, and gives the security administrator a user interface that they can use to slice and dice the data at will.
Event logging is, of course, one of life's irritating compromises. Author Mark Twain is quoted as saying that: “A classic is something everybody wants to have read, but no one wants to read”.
Here in 2016 we say that system log files are something that we all want to have available but none of us actually wants to have to weed through or find space for. Servers aren't all that bad – at least disk space is cheap and it shouldn't be too much of a problem to find a bit of storage to retain sufficient log data to allow you some measure of after-the-event analysis.
But for network devices – routers, firewalls, switches – storage is at an absolute premium, and you can seldom keep anything like enough detail in the log files. Turn down the logging level and you can probably store several days' data but at a level of detail that's so sketchy as to be utterly useless. Turn up the logging to a level that's actually useful and its longevity will be roughly similar to the average particle that's discovered at CERN. The solution is a central system with enough storage and power to let you hold the data for a sensible time and report on it at will.
Size and resilience?
As we mentioned, a SIEM system is in principle a single point of failure. You need a socking big database to keep the data in, as you'll want to turn the logging levels up nice and high. And you'll also need a meaty processor and plenty of memory because you'll be processing these big lumps of data at two levels: first, you want to run reports on it which means potentially complex database queries behind the scenes; and second, the SIEM system should be working in the background collating the various streams of data it's receiving in order to spot commonalities and inter-relationships between the streams.
It needs to be big, then. But what about resilience? Well, it's kind-of up to you, but let's think about it for a moment. We've said that you can store a reasonable amount of log info on your servers but bugger all on your network devices … but what is a “reasonable” amount? It's generally reckoned that the average cyber incident takes upwards of 90 days to detect on your network – and reports I've seen cite figures such as 140+ days or even a year.
A well-hidden intruder (or more likely a well-hidden piece of malware) can nestle undetected, quietly sending your data to its illicit external location. Realistically, then, you really need a means of storing the data for up to a year – which probably means it's not going to be realistic to crank up the server log level and store it locally.
The point of this last comment is that the SIEM system isn't storing a copy of the log files: it's storing the only instance of it. If you want to look back ten months in the logs, they're only in one place – which means if a single-instance SIEM system craps itself you're in trouble. So either make sure you have a very strong backup in place for the SIEM system or, preferably, make it resilient just like the infrastructure.
The hybrid world
At which point we get into the first mention of the 'H' word. I'm writing this piece not just in the context of SIEM itself but specifically with SIEM in the context of a hybrid cloud – where part of the infrastructure is under your control in your own office or data centre but the rest is sitting in a public cloud provider's setup.
In a hybrid setup you have options as to where you put the SIEM system. You could implement it as an in-house setup – either with physical appliances or on your virtual infrastructure – or you could put it in the public cloud as a virtual setup. If you go for the virtual option you have to be ultra-cautious about making it scale (even a modestly sized organisation can be throwing 1,000+ log messages per minute at the SIEM system, which has bandwidth and processing overhead implications) but this is surmountable as long as you do it properly.
SIEMing your private cloud is pretty straightforward, since all the products on the market have been built in the knowledge that this is what people will want to do. So they know how to probe WMI streams on Windows server estates, and they'll eat as much Syslog data as you wish to feed them with. Now, with the public cloud component of the hybrid cloud the same concept applies – a Windows box in the public cloud is no different from one in your private setup as long as you've configured the connectivity so that the SIEM system can talk to it.
The main problem, as always, is that in the public cloud you don't get visibility of all the layers of the infrastructure. In your private world it can deal with everything from the physical port up to the presentation layer, but in the public cloud things kind of stop when you get down to the virtual switch and virtual port level.
When you reach this point you need to do a bit of research and evaluation, because both the SIEM vendors and the public cloud providers are working in parallel – and with each other – to help fill this gap and you' need to find the best combination of cloud vs SIEM.
Although in some cases you'll have a private setup with SIEM and will be looking to expand into the public cloud, in most cases it'll be a hybrid setup already into which you want to introduce SIEM. And if that's the case, it won't take you long to find which SIEM packages are available with specific support for your cloud provider (a couple of minutes in the AWS Marketplace will throw up some good options, for instance, if you're an Amazon cloud user).
But even if your preferred SIEM solution doesn't have specific support for the pubic cloud of choice that's not a show-stopper – because as long as the public cloud setup can throw its low-level event logs at you in a form that the SIEM product supports, that's often enough.
Where to put it
I've mentioned that you can choose either the public or private cloud for the SIEM system, but there's an important point to bear in mind: you wouldn't necessarily put it in your existing public or private cloud.
The value of SIEM is that you can use it to analyse what happened in a security incident. Now, that incident probably involved the intruder copying or corrupting data, or causing some kind of outage. So if the SIEM installation lives in the same installation as the systems whose logs it's consuming, it's susceptible to the same attacks and there's every chance it could be killed off as part of the attack.
In a private setup that means that at the very least you need to host the SIEM product on separate hardware in separate cabinets from the rest of the infrastructure. Ideally you'd have it in separate premises, but there's a hefty cost associated with that.
The hybrid cloud actually gives you an opportunity that you don't get in an entirely private setup, then: a low-cost chance to put the SIEM system in a different public cloud installation from your core systems. Most of the time this will be a different part of your public cloud provider's installation – same provider, different region – as you'll generally consider it reasonable to have physical separation without the need to go all the way to using a completely separate provider.
What you must be sure of, though, is to have the SIEM installation separately managed as well as physically and electronically separate – the interconnect should be extremely restricted to permit only the log traffic (and management connectivity) that you need in order to use it, and you should particularly not allow management connectivity from the production public cloud into the SIEM public cloud, in order to prevent intruders from destroying your forensic evidence directly from the production world they just compromised.
So, ironically …
The hybrid cloud does have a few idiosyncrasies you have to overcome – primarily the fact that implementing SIEM in the underlying infrastructure will require a bit more jumping through hoops than it would in the private elements of your world. But these can be overcome and in fact the hybrid setup actually provides you with some genuinely useful mechanisms for maximising the robustness and security of your SIEM solution, and for doing so more cheaply than would be possible in the private setup. ®
Sponsored: Becoming a Pragmatic Security Leader