Better governance: banking on continuous delivery

Senior Director at Capital One, Tapabrata Pal, discusses “Better governance: Banking on continuous delivery” at the DevOps Enterprise Summit 2018

Pal has more than 20 years’ of IT experience in various technology roles (developer, operations engineer, and architect) in the retail, healthcare, and finance industries.

Over the last six years, he has evangelised and led the company’s DevOps initiatives. He is currently Senior Director and Senior Engineering Fellow focused on DevOps, and continuous delivery at large scale in a regulated environment, and is also the community manager and core contributor to Hygieia open source project while helping Capital One build their own software on the public cloud, implementing continuous delivery and microservices.

Public cloud software

“We are the first bank to put all banking applications on the public cloud”, revealed Pal.

“We’re an open source-first organisation — actively using, contributing and managing open source software projects.”

According to Pal, over the last 5 years, the bank has made many, many changes: Waterfall became agile; manual builds became automated builds; manual deployments became automated deployments; manual tests became automated tests; its data centre became the public cloud; and closed source first became open source first.

“Among the changes, several things happened. We switched from outsource to insource, and from vertical silos to production teams – DevOps, QA, Release Managers to Engineers,” commented Pal.

Capital One’s goal!

Last year, the bank’s goal was to “slay the monolith” and to have “no fear releases”.

”There are a lot of things which are unknown, so it can be scary releasing a product. Especially as it’s your responsibility to ensure that everything is working if you built and tested it. As soon as you deploy it, you own it,” added Pal.

“There’s always the fear of speed, fear of being out of control and fear of being non-compliant.”

Nevertheless, at Capital One, they want to release products as soon as they can, safely, with governance being key.

Pal continued: “Governance is a good thing. Compliance is checking the box, and we have three lines of defence – who owns the risk? who set the policies? and who monitored the risks?

“The developer’s role in governance when doing continuous integration (CI)/continuous delivery (CD) pipelines means they must be extremely aware of what’s going on in their product, as well as implement controls to help; although it can feel like someone is dictating your work when using controls.”

Continuous delivery controls

Despite this, controls can protect you and your company, provide assurance around financial reporting, and are useful towards investments.

“At Capital One, a minimum of two controls are needed for CD, as well as two pairs of eyes. It’s also important to ensure that you don’t change anything without authorisation.

“You can build on every code when automating, but the biggest hurdle is ensuring single developers don’t make changes to production by bypassing all code.”

Pal revealed that he finds using “Clean Room” useful, which is a software engineering process and software development process intended to produce software with a certifiable level of reliability. Its focus is to process defect prevention, rather than defect removal.

“Clean Room is an environment typically used in manufacturing including pharmaceutical products and scientific research, as well as a semiconductor engineering application with a lower level of environmental pollution such as dust, airborne microbes, aerosol particles and chemical vapours,” he added.

‘Software Delivery Clean Room’

According to Pal, by the end of 2018, 50% of Capital One’s applications will be in the Software Delivery Clean Room.

He also noted that, if you’re doing CD, Clean Room will show you have good data and are better managed.

Written by Leah Alger