The rise of DevSecOps

With ever-increasing security threats, which are becoming more and more complex and sophisticated every day, many enterprises are opting for DevSecOps approaches. DevSecOps integrates security within its operations and development, so as businesses are protected in the best way possible from the start.

Our current climate has seen the evolution of security threats and how important having a good system incorporated into the development lifecycle from the very beginning. It is then good to ask ourselves if DevSecOps is the future of DevOps?

To shed light on this topic, we asked experts in the industry to share their insights.


Why is DevSecOps important?

Taya Rybak, NWE QA Product Owner Cloud Team at Deloitte, defines DevSecOps as ‘the practice which advises companies to incorporate security principles in the development and operational process.’ In our modern cyber world, the security of data is becoming the top critical aspect for companies. She believes that DevSecOps practice leads people to think more about security for all application components through the whole life cycle. While in traditional development, security could be forgotten or only be the responsibility of one single team.

‘DevSecOPS shows that each level of application is important to guaranty security and all teams have to be involved.’

Mark Schumann, DevOps architect at Blue Herring, also adds that DevSecOps is not a new idea. On the contrary, it is a revival of traditional practices that didn’t have a label until now.

‘Before there was DevSecOps, before there was DevOps before there were really even “ops” to speak of—we didn’t have a name for it, we simply didn’t have the means to separate security from operations from development. If you wrote code, it was your job to figure out how to deploy it and to do so securely. Professionals were expected to know security and operations and code.’

However, no one wants to go back to those days, Mark says. The tools were ad hoc, the procedure unreliable, and the version control informal or flaky. Everything was too prone to error and none of it scaled very well.

The point, he continues, ‘is that software development professionals have never not been proactively solving problems in operations and security’. What is changing now is that there is a new spotlight on DevSecOps and that responsibility is being acknowledged recognized, and supported.

According to Mark, DevOps and DevSecOps are now recognized as disciplines, with conferences, communities of practice, and well-supported tools.

Moreover, Ugo Ciracì, project lead at Agile Lab, emphasizes that the rise of the importance of digital data in our society puts everyone in a vulnerable condition.

Indeed, now that we delegate governments, social networks, digital services to use our personal data on our behalf, we have to agree to terms and conditions about the data privacy and usage of personal information.

‘Direct and indirect information that a digital service can grub about our life is huge and it is hard to control,’ Ugo continues. ‘On the other side, as a software house or factory providing digital services, we must comply with some regulations to guarantee that the personal data we manage are safely stored, carefully handled, and protected.’

In order to keep up with all these requirements, we need to have a “paradigm” in place. Hence, we need rules, automation, a culture about how software must be developed and delivered as well as being able to monitor and control software security continuously.

‘Taking together all those aspects is what DevSecOps addresses today’, he highlights.

According to Ugo, cybersecurity affects every single step of the software lifecycle whether it is for a mobile app or machine learning factories. In testing, security spans the whole journey of a software artifact.

Thus, in this perspective, we can consider both testing and security to be the biggest result within the DevOps movement since security is no longer confined to a silo within a company but is distributed across all the steps and roles of the value chain from development to release and documentation.

DevSecOps helps cybersecurity costs of development to get lower and controlled.


How can DevSecOps improve software development and delivery?

As Mark points out, DevSecOps is systems thinking applied to the whole lifetime of a software system, rather than arbitrarily stopping at the point when the code works. With the evolution of the landscapes and tools, implementing DevSecOps offers two big benefits:

  • Integrating for productivity: when an organization has security fundamentally integrated with development and operations, they won’t hold up a deployment whenever the “security person” raises a red flag at the last minute. Hence, DevSecOps means everyone designs for security, all the time. Those red flags appear earlier on, and they don’t have to be as disruptive.
  • Integrating for security by starting with security awareness: Under DevSecOps, businesses build threat monitoring into your code. To do that, they need to verify their dependencies and build security checks into their unit tests. Thus, when security incidents are discovered, DevSecOps means closing the loop by bringing them into your sprint retrospectives.

Mark also highlights the fact that DevOps practitioners usually talk about ‘shifting left’, meaning that businesses are going to build testability and deployability into applications so as work done earlier in the process supports the completion of the process. DevSecOps takes that farther and shifts security to the left too.

Ugo also emphasizes this point. Indeed, DevSecOps allows us to address different problems leveraging the shift-left paradigm from development to automated releases across all intermediate steps and environments.

He continues by pointing out that software engineers, test engineers, cybersecurity specialists, release managers, operations, automated pipelines are all responsible for specific duties on a given step. For instance, it would be like a developer applies defensive programming techniques, unit tests check for basic flaws, automated code-quality tests seek for weaknesses through SAST, all the while cybersecurity specialists run penetration tests of any kind and the production environment leverages some sort of peripherical security like a WAF.

‘Every stage determines costs.’

Ugo highlights that distributing the duties of security reduces costs since we don’t have to wait for the latest step to discover that there is a flaw that could have been detected at the first stage. The DevSecOps approach allows for inspecting many perspectives and producing more secure software.

Taya also adds that DevSecOps help companies build secure application faster by starting security validations from the early phases. In traditional delivery, security issues can be found just on Test Phase, which would make it expensive to fix and the whole application might have to be redesigned. If a company has DevSecOps process security controls incorporated at the beginning of the delivery instead, then most issues are found and fixed earlier in the process.

Therefore, DevSecOps brings more productivity and security.


Slow adoption

Mark underlines that shifting left is hard. Otherwise, everyone would be doing it already. Adopting DevSecOps is a challenge and not every organization is ready to overcome that challenge.

‘The best we can do is to bring about the conditions that make that adoption more likely and more sustainable.’

Based on her own experience, Taya highlights that some companies’ delivery processes are not mature enough, and so, when they start to implement DevSecOps on top of it, they are usually faced with multiple challenges and that is only causing additional issues.

Moreover, Ugo adds that ‘DevOps is not for free and adding any dimension (security, data, legal and compliance, development, operations, etc.) requires a company to be ready for a mindset switch where reorganization, new tools, new skills, and a new culture are necessary.’

He also points out that the most common mistake that a company can make is to create a “DevOps” department in charge of all CI/CD automation to consider they are doing DevOps. This is how DevOps dies: by creating a silo among the others.

According to Ugo, adopting DevOps is a path of improvement to take iteratively and incrementally. Collaboration is a key factor in order to succeed. Further, a good starting point would be to have everyone involved in the software lifecycle is open to take new responsibilities, review processes, earn skills.

It’s not only about security, he adds. Indeed, security can’t take off without automation, testing, and monitoring. There are several decisions to take, among which security is a big concern. An in-depth security assessment must uncover the insights a company can match against its own risks. Yet, it is fundamental to identity which concern mostly affects risks of the company: it may be legal and compliance or business continuity. It’s a matter of risk assessment.

‘Sometimes, the reason why DevSecOps is slow to return the investment is due to a lack of clarity in the goal to reach.’


Implementing DevSecOps

In order to implement DevSecOps successfully, Mark emphasizes that the main thing is knowing how to keep score.

Indeed, teams that are evaluated on ‘velocity’ will optimize velocity. So, you need to define velocity in a way that includes the Ops and Sec part of DevSecOps. Hence, the best way to adopt DevSecOps practices is to place a value on the entire software development life cycle. You need to be aware of what you are rewarding because that’s what your team will focus on.

To adopt DevSecOps practices, Taya underlines that the company should first have a DevOps process or Continuous Integration. Otherwise, moving to  DevSecOps might be a problem. She then suggests to start with stabilizing the delivery processes: introduce continuous delivery, identify quality metrics, and after the company gets reproducible and measurable process, implement DevOps

Ugo points out that to implement DevSecOps, it is important to ‘build a shared path where all the stakeholders are involved, goals are clear and understood by everyone and there is transparent and genuine participation.’

Depending on the DevOps maturity level of each organization, there are different approaches. For instance, some companies are completely affected by internal politics. These cases are challenging to manage and even if decisions come from the top, this situation may lead to failure in a short time because of non-collaborative environments.

Thus, the incentive of many kinds to smooth corners is important, at the end the world is moved by humans.

Besides, there are cases where a small and cross-functional team can provide a full set of security features within a DevOps context. This is the lucky case as you can represent the value of your DevSecOps process.

Ugo adds that there are green/brownfield approaches. They are both valid depending on the situation. Usually, a brownfield approach is more difficult since there is a need to make a switch in the middle of a running situation, which requires less pressure. Meanwhile, a greenfield approach could better embody new dimensions within the DevOps process. Yet, what often happens is the opposite. On-going and relevant projects are the ones that can show the real impact so that at the end they get more pressure and expectation about results.

Finally, Ugo recommends finding a core group of genuine and skilled people that can move the gears from the top to the bottom of the pyramid to promote and implement the initiative.

DevSecOps practices

According to Mark, the most important DevSecOps practices overlap significantly with Agile practices and DevOps practices.

For instance, if you’re doing Test-Driven Development (TDD) already, you need to think more about the scope of your test cases. Do they take into account a realistic threat model? What dependencies do you have to mock out? What are the security implications of those dependencies? If you’re already doing Continuous Deployment (CD), shouldn’t static analysis tools be built into the pipeline? If you’re using containers, where are they being deployed and how do they need to be restricted?

Moreover, he continues by saying that you need a consistent monitoring plan, even if it’s a simple one to start with. Start by analyzing each significant system, documenting how it works in production, listing all the external and internal resources it relies upon, and then documenting how state changes in those resources can affect the success of that system.

It’s just another way of asking “What could possibly go wrong?” The answers to that are your checklist for monitoring and remediation procedures, and you can feed that checklist back to the development stage to prevent problems instead of always reacting to them.

Mark also stresses that there are so many options in DevSecOps tools, as the Sec part is all about your operations context and the conditions of the enterprise. Understanding the threat profile is priority. Financial, aerospace, military, games, manufacturing, IoT, and customer service environments face varying hazards and require customized solutions. Many shops start with code analysis, intrusion detection, and configuration change monitoring.

Finally, Mark recommends working hand in hand with the security professionals who already know your environment

On the other hand, Ugo points out that the best tool is the one that fits your needs and goals. It is important to not overlook the reality of things. For instance, ‘consider that OSS is one of those realities. Many de-facto standards come from open-source communities and they are often embedded into commercial software that wants to keep pace with the nowadays dynamic. Think about Kubernetes, Spark, Spring-boot, and many other libraries, systems, and frameworks that are spread everywhere. They compose most of the custom software we develop. Thus, they are the major source of security problems (and solutions) we have. They are a problem because we are users and clients of those OSS artifacts so that any flaw met at that level is likely to affect security at a worldwide scale.’

They are also the solution; their popularity and broad adoption bring communities to actively cooperate and work together to find a solution.

Moreover, Ugo says, it often happens that companies spend a lot trying to protect from their own weakness, which only represents a minimum part of the real codebase they are using, while software composition analysis would show how much they should care about libraries they are leveraging on.

On the one hand, open-source code is 100% available to malicious people as well as to ethical hackers, thus it is presumable that they are going to “attack” that code more easily. On the other hand, internal protection is required to prevent internal unintentional mistakes and intentional attacks.

Therefore, ‘good development best practices help to moderate these risks. Code-review, pair programming, code quality, etc. are all good ways to reduce the impact of mistakes and malicious insiders.’


The challenges of DevSecOps

One of the main challenges of DevSecOps, for Mark, is when management brings new tools or methods but doesn’t actually support or reward an actual change in process, the improvements then tend to be superficial.

To avoid this, Mark recommends looking for concrete improvements that can be made in the next six weeks and the next six months. Document their value and devote resources commensurate with that value. And not to accept intransigence at any level.

Ugo adds that DevSecOps inherits all the challenges of cyber-security in the new era of fast-paced developments. Indeed, having cross-functional teams is one of the most important challenges to face. Tools and automation can lead the process, but humans still need to regulate everything.

‘The many layers from software to infrastructure and hardware require a deep understanding of techniques and technologies, interactions and integrations, networking, security, development requirements.’

Serverless computing achieves a high level of abstraction for a software developer but also hides a lot of complexity. This is challenging as whenever a data breach happens, you need to detect how much it affected your customer base. Therefore, having a cross-functional team covering much expertise, skills and competencies can do the difference. Again, human capital is a true investment for DevSecOps.

For Taya, the biggest challenge is to move the Development and Operation Teams to the proper maturity level and to explain to management that in order to implement DevSecOps, the company has to invest time and money in improving delivery processes.


DevSecOps: future of DevOps?

Mark believes that DevSecOps is a learning process that should lead to decreased risk and more consistent delivery of business value.

‘Like DevOps, DevSecOps is a mindset that won’t become outdated and undoubtedly has a place in the future of computing. How your team applies that mindset should depend on its milieu and problem space; it will always be about adapting and iterating.’

Ugo believes that DevSecOps together with Agile and DevOps is undoubtedly the present. However, he thinks that they will soon be outdated once they would have become part of the common obvious culture of software system developments. But then, we will be able to enjoy other challenges.

‘We are in a complex world where the shift ahead is necessary but there is not yet a theory of everything that simplifies the vision. The big challenge with DevSecOps and its cousins will be to find the way for harmonization where automation, culture, lean principles, and gauges will be no longer under the pressure of groundless prejudices. Today the impediments come from people, not really from technologies. Agile principles are difficult to apply. As such, also DevOps suffer because of a lack of a continuous improvement mindset. Once we will digest all those things as if they were the normality, we will start feeling the result of investment in people.’

Finally, Ugo ends by saying that cyber-security is an impacting issue in our daily life. IoT and wearable devices, smart home, social networks, digital services affect our safeness more than we realize. ‘As persons, we delegate our security to software and electronic systems, we let those systems store our private data in the cloud. As a user, I would be more than happy if my service providers and device suppliers cared about my security and privacy.’

He warns that the next step to be a user and not a “used” should be transparency about the guarantees and quality. He believes that an effort from all communities should go in this direction.


Special thanks to Mark Schumann, Taya Rybak, and Ugo Ciracì for sharing their insights on the topic!