DevOps: Bloody hell, we've got to think about security too! Sigh. Who wants coffee?

How to bake in security to DevSecOps, er SecDevOps ...

By Dan Robinson


Imagine you're an organisation that is looking to implement a DevOps approach to applications and services, or perhaps you’ve already started, but you’re worried about security.

DevOps is all about rapid iteration and continuous delivery, but your security folks still want to be able to do checks to ensure systems are as bulletproof as possible. They likely to make sure your organisation is meeting regulatory requirements like GDPR, which comes into force very soon. How do you fit in security while staying agile?

OK, but we're deploying all the time...

Security can no longer be ignored or sidelined – just look at the damage caused by the WannaCry ransomware outbreak last year, for example - yet according to a June 2017 Global Security Report by Trustwave, 99.7 per cent of web applications* tested by the firm had one or more vulnerabilities.

If you’re looking for shortcuts and easy answers, then prepare to be disappointed. Security is difficult, and making it an integral part of DevOps – whether you call it DevSecOps, SecDevOps, or something else – is about having the right tools, having the right processes, and about cultural transformation. The good news is that if you have already had a measure of success in implementing DevOps, you should already be well prepared to make security part of the mix.

Who is assessing the 95 per cent of third-party code you’re using?

There has been a growing level of interest in DevOps over the past several years, as businesses started to realise they have to become more agile in order to maintain a competitive advantage. The result has been a drive for closer collaboration between developers and IT operators, in order to quicken the cycle of development and deployment.

Security needs to be treated the same way. The traditional way put security assessments towards the end of the development process, before an application went into production. If this held up deployment for weeks or months, it may have been acceptable when development might take a year, but with DevOps and continuous integration/continuous delivery (CI/CD), you may be deploying a new release on a daily basis.

This means that security needs to be involved right from the start of development. Secondly, security checks need to be automated and embedded into the development process itself, and thirdly, there needs to be auditing and monitoring to ensure that the correct processes have been followed.

You can't just dump a long list of requirements and run...

One person that has actually overseen a move to DevOps in the past is Dominik Richter, engineering manager at Chef, developers of the eponymous configuration management tool. He also helped to create Inspec, an open-source tool for automating compliance checks.

“To get security into the mix, first of all you bring them to the table. It is very important that you start with conversations, bring people together and actually get them together and talk. It isn’t enough to just come in once a year and drop a bunch of requirements on the folks in operations and devs. You have to trust to collaboration first,” he said.

Getting security involved right from the start means that many flaws will be picked up at an early stage in the development process, rather than later when all the work has been done and would cause delays to go back and fix things, Richter adds.

One tip that Richter offers is to put security requirements into the project documents and then turn these into code, in a similar manner to the way infrastructure and application requirements can be described using Chef recipes or Puppet files.

“Like you describe the end result [in Chef], so someone else can take that and run it and get the same result, the same thing happens here; you describe the security policy in code, which is what we did with Inspec,” he said.

A similar idea is championed by Mike Bursell, chief security architect at Red Hat, based on ideas presented by his colleagues at the OpenStack Summit in Sydney last year.

“One of the things they looked at is taking requirement documents, such as those you get from the government, for example, and deriving those security requirements and making Bugzilla (or whatever bug-tracking system you are using) entries for each of them.

“Then, as you go through your development process, each cycle you can say ’You know what, I’ve met this particular security requirement, but I’m missing these 99 other ones.’ That’s fine, because it’s your first go at it, but next time you go around, you are fixing it,” Bursell said.

Automation is also key. To ensure that checks do not hold up the development cycle, these need to be automated and performed whenever a developer makes a commit to the code, before it goes through into the production environment. These could be custom checks or a SCAP check or invoking off-the-shelf security tools, but the important point is that it is an automatic process that is embedded into the development cycle.

“In order to get the speed, agility and flexibility from DevOps, you need to automate everything. You need to automate build; you need to automate test; you need to automate regression testing; and I believe that test automation benefits security coming in to the DevOps process more than anything else,” said Chris Carlson, vice president of product management at Qualys, which develops vulnerability scanning tools.

However, Carlson cautions, organisations need to make sure that the process is performing the right functions, and not just serving as a tick on a compliance checklist.

“Automation is just a mechanism to execute a process without human involvement. But the process needs to be correct, and it needs to be monitored and continuously evaluated and improved over time, because if you automate an incorrect process, you’re going to create more problems,” he said.

Define, measure...adjust, refine

For this reason, some dev teams start by defining the process, perform a manual execution at first, then adjust, measure and refine, and only when they are sure it is delivering the right results do they move on to automating it. It may also be a good idea to train up your developers on secure coding principles and how they can be best applied, but Carlson says that this is only part of the solution.

“That puts an undue burden on the developer to know everything that could happen from a security point of view in what they are building, and that’s tough,” he said.

This is especially so when you consider that most applications and services are actually assembled from existing components rather than built from scratch.

Developers take Java or PHP as an application execution framework, then import open source libraries to fulfil functions such as visual graphics or data processing, so as little as 5 per cent of an application may be custom code.

“Who is assessing the 95 per cent of third-party code you’re using? You often don’t have access to the source code, and even if you do, who is going to review a million lines of code in the Apache Web Server? That’s why you need to integrate vulnerability scanning tools like ours into the dev process, the QA process, the ops process, without them knowing that they are using security tools, so they don’t have to be security experts,” Carlson said.

That said, part of the cultural transformation process is not just getting developers, ops and security to work more closely together as a broader team, but getting everyone to take equal accountability for defects, whether they are functional defects, quality defects or security defects, he added. Otherwise, it is too easy for developers to simply throw security issues over the wall to the security team instead of getting on and fixing them.

“Common security attacks like SQL injection and cross-site scripting are input validation errors that developers fix as a matter of course, but in organisations that have been successful in implementing DevSecOps or security into DevOps, you see a cultural transformation where they treat security defects the same way they do other defects”, Carlson said.


One big worry for organisations is the EU’s GDPR legislation, which takes effect on May 25. Among various things, this includes new data-governance obligations covering privacy and protection of personal data, with hefty potential penalties for breaches. Many firms fear that they may be blamed if a breach is caused by insecure code or systems.

The good news here is that GDPR compliance is also about having the right processes and adopting a culture of privacy, and it seems that many of the processes that businesses need to put in place in order to deliver a successful DevSecOps strategy can actually help, according to Bursell.

“I think this is where, if you can show that you’ve got your governance in place, which includes GDPR requirements, and you’ve derived policies, and you’ve automated them and you’ve got your monitoring and you’ve got your auditing, then you’ve gone a good way towards being able to show that you meet GDPR compliance,” he said.

“Any auditors that come in are going to want it demonstrated to them that the policies you have shown them on paper or on screen are baked in and can be tallied with artefacts from your process, which is why you need the auditing and the monitoring,” he explained.

So there you have it. Security is a non-trivial issue, but good security practices as part of a DevOps strategy are no more difficult than for traditional waterfall development. Rather, the processes and cultural transformation that you need to deliver a successful DevOps strategy also make it easier to deliver better security compliance by design as well. ®

* derived from over 4,000 penetrations and 10,000 + scans over the course of a year. From small biz to ents

We'll be covering DevOps at our Continuous Lifecycle London 2018 event. Full details right here.

Sign up to our NewsletterGet IT in your inbox daily


More from The Register

CLL '19 to span DevOps, Containers, Continuous Delivery and Serverless

Events Agenda is set, blind bird tickets going soon

Save £100s on DevOps, Containers, Agile and Continuous Delivery NOW

CLL19 blind bird ticket offer expires soon

40 reasons to save big with CD/CI, DevOps, and Containers

Events CLL19 Blind Bird offer expires soon

The DevOps Salary Report is in, and is great news for American men

Want some more cash? Get some more skills. And perhaps a moving truck

AI, Reinforcement Learning, Neural Networks... DevOps? Learn more this month

Event Making machine learning work at work

Microsoft squeezes a 2019 server out of the Azure DevOps pipeline

RC1 is here for devs without their heads in the cloud

Puppet Insights arrives to shine uncomfortably bright light on DevOps

Want to know if all that cash you spent on consultants is paying off?

Why did Visual Studio Marketplace go down in the Great Azure TITSUP? Ask Azure DevOps

Failover is not an option

Visual Studio Team Services squeezes into new Azure DevOps togs

Azure here, there and everywhere. Except last week when VSTS was nowhere