By: Nishant Sashidharan

DevOps is a hot topic and every enterprise is trying to implement it to stay relevant in the industry. When I was introduced to DevOps I was highly skeptical that it would work because we were talking about over 40 deployments in a day and with lead times that were about 200 times shorter. I was deeply concerned about the security and robustness of these builds being deployed as the only artifacts that supported this was from smaller companies and there was nothing to prove it worked in large enterprises. That was my jittery start with DevOps.

Then, the closer I got to DevOps, I felt there was nothing new as I grew up coding using XP and the practices that were in DevOps such as dark launching  were just a part of the things I did during my XP days.

So, I had to dig deeper past the hype to see the value and calm my concerns about security. I pleasantly found excellent ideas and patterns that could add value to the way we can build robust, secure and risk-free systems. The one thing that got me to my first “aha” was the proven ability of systems to recover almost 150 times faster than normal.

We are going to explore some of the important things that we can and need to do to achieve these kinds of results by enhancing security and enforcing compliance all at the same time. Sound interesting?

Before we get into the need and details about DevSecOps let us quickly refresh the advantages of DevOps along with a few challenges.

1. Speed of Delivery - One of the main reasons companies are adopting DevOps is to increase the speed in which they can bring change to the market. Unfortunately, with speed comes a great responsibility of quality and security. How do we address that?
2. Design and principle of YAGNI – DevOps is built on the principles of agile, lean and YAGNI (“You Ain’t Gonna Need It”)  which means that we do not have any large upfront designs and we are constantly cutting down our design and features to meet only the most important thing. When and how will the security teams’ model for threat with such constant changes?
3. Microservices & Containers – DevOps introduced us to the world of designing small, isolated functions that can be changed, tested, deployed, and managed completely independently. Unfortunately, it also comes with some downsides like operational complexity, polyglot programming, logging strategy and attack surface. Looking at containers, especially the famous Docker, when used with microservices can actually provide micro segmentation. Adrian Mouat author of ‘Using Docker’ has stated some serious security concerns that come as a part of Docker such as Kernel exploit, Denial of Service attacks, Container Breakouts, Poisoned Images and Compromising Secrets. How can we have security cater to all of these areas with confidence?
4. Compliance & Change Control – Adapting to DevOps brings in the need to change the old compliance and change control process where there is a mention of SoD (Separation of duties), clear handoff and wait time to ensure that requirements for data confidentiality and integrity are satisfied, sothat the data and configuration cannot be altered by unauthorized individuals. How do we address this while moving fast?

The answer to all of the problems above lies in the magic of DevOps + Security or DevSecOps. I will explain some ways we have been able to solve these challenges by wiring security and compliance into DevOps.One of the most important things to start our journey into DevSecOps is to Shift Security Left. We need to introduce security earlier into planning, design, coding and into automated test cycles instead of doing it at the very end like we do today just before the release. 

A great example for this is the popular story of DevOps at Etsy. Etsy’s security culture is built on top of its engineering culture and connects with the entire culture of the organization. The driving principles for this are:

1. Trust people to do the right thing, but still verify. Rely on reviews to prevent or catch mistakes. The production mistakes go through a process of blameless postmortems to examine and understand the source of the problem and fix things at the source.
2.  “If it Moves, Graph it” – Make data visible to everyone in the company so that everyone can understand and act on it including information about security risks, threats and attacks.
3. “Just Ship it” – Every engineer can push code to production at any time. This included security engineers. If something is broken fix it and ship it right away. Send it through the CI/CD pipeline so all the tests are done again to the fixed code.
4. Security cannot be a blocker – The word “No” is a finite resource – use it only when you must.

Etsy’s security team also takes a number of steps to build relationships between security and engineering so that they blend into the overall culture of the company.

The next important thing to follow to ensure having security built in is to follow OWASP proactive controls which is a set of development practices, guidelines and techniques that we should follow to build secure applications. These practices help shift security left by injecting security into planning, design, coding and testing. When we start following these practices, we are in turn creating the process of making Security as Self-Service. We want the teams to be able to inject security seamlessly the same way we provide automation or virtualization. For example – At Twitter, security checks are run automatically every time a developer makes a change. They use tools like Brakeman to the check the code for security vulnerabilities and provide feedback directly to the developer if something is found. Developers also have a “bullshit” button to reject false positive findings.

Another key to DevSecOps is Infrastructure as Code - managing infrastructure configuration in code using tools like Ansible, Chef, Puppet, and Salt. This speeds up and simplifies provisioning and configuring systems, especially at scale. It also helps to ensure consistency between systems and between test and production, standardizing configurations and minimizing configuration drift, and reduces the chance of an attacker finding and exploiting a one-off mistake. Security teams can also use Infrastructure as Code to program security policies directly into configuration, continuously audit and enforce policies at runtime, and to patch vulnerabilities quickly and safely. A good example for this is a service called UpGuard.

The bottom line is to continuously build security and compliance into the code while building the CI/CD so that when you are ready, security or compliance is not going to a stage gate to stop the flow. In the organizations that I have been engaged with, moving to DevSecOps always starts with DevOps and continues as a journey making small changes along the way. It requires new skills, open thinking, new tools, and a set of new priorities. So, start small but start soon! Be safe!