In DevOps, and pretty much every Lean-Agile methodology, when speaking of Flow, it is referring to the end-to-end production chain of software, from ideation, through development and testing, into the Production release, and extension of that to maintaining and sustaining it in a reliable, high performing state.
The DevOps “Mission Possible”
The first step is to establish a Mission for the DevOps team to target and follow.
In order for the team to properly visualize a Common Vision and work towards establishing that, is to equip them with a Common Mission, which would allow them to identify the work that needs to be done and enables them to properly prioritize their work and distinguish among different classifications (from 911/Firefighting work items to business-as-usual).
Since DevOps teams come in a variety of skill-set compositions, and they serve a variety of different lines of businesses across the enterprise or market sectors, we need to customize their Mission Statement to match what we have envisioned them to work towards.
Their tailored Mission Statement should also reflect their organizational compositions, challenges, and goals and their relationship with their internal and external customers.
Mission Statements would become clearer and provide a better assessment of the current standing of the DevOps teams when they provide data on where the team is in a position against each goal listed for them to achieve.
For example:
Over the past 3 months, we have achieved an Average Lead Time of (….) and we want to reduce that by a total of 70% to an Average Lead Time of (….).
We want to achieve this by the following target reductions in our Work-In-Progress sections’ Through-put as per the following:
65% Reduction in Through-Put for Development Section (from current …. to ….)
80% Reduction in Through-Put for QA Sections (from current …. to ….)
Mapping the Value Streams Across the Enterprise
Now that you have given the DevOps teams their own tailored Mission Statements (or target Metrics) to achieve, it is time to identify the Value Streams that they are part of.
Value Streams are the series of steps that an organization uses to Turn an Idea into a Value and deliver that to the customers.
To get more granular, Value Streams are the sequence of systems, people, tools, and processes that have lined up together to get that Idea and turn it into a Product or Service (value to the customer) and deliver it.
There are Three types of Value Streams:
1. Operational Value Streams
These are the steps (or sequence) that provide the Products or Services to the Customers and generates Revenue.
Like the Mortgage sales depart of the bank which the chain of people and systems and processes that are in play from connecting to a mortgage applicant, creating a profile and application for them, doing the credit check, and deciding how much can be provided to that applicant, issuing them the approval, getting them the fund and the start collecting that back with the interest.
Operational Value Streams don’t create the Product or Service that they sell. They are responsible for selling them to the customers and collecting the Revenue.
2. Development Value Streams
These are the steps (or sequences) that actually create Products or Services that are being sold to the Customers. They are also responsible for creating the tools that the other value stream uses in their activities (like the customer registration system, credit background check system, financing system, and all other tools they use).
3. Dual (aka Combined) Value Streams
The border between these two value streams gets unclear in Startups and smaller organizations since they sometimes use the same team for both of the operational and development streams, as they both create the product for sale and then directly present it to the customers.
This is more like a collapse of the other two into one, and good practice would be to try to separate them and give them their own distinct domain of activity at the earliest possible window, so each stream can specialize deeply into their field.
Value Stream Maps
In order to help the DevOps teams, build the needed insight into the steps of the Streams and to help them identify the potential areas plagued with inefficiencies and constraints, you can provide them with the Value Stream Maps.
This would be a “Just Enough” detailed map which will show our workflows and steps involved in each area of work.
This map shows each step and its relationship to the steps before and after it. It also pictures the Throughput that is expected of that step and shows the actual number that we have observed and measured.
It will also carry any additional needed information that could help the DevOps teams better target the troubled areas (e.g., the # of rejected items, or #or bug found, or any specific reason that caused the delay).
DevOps can target inefficiencies anywhere within their Value Stream and flow of work. It may be directly related to their own delivery pipeline, and it may be related to the systems they build and maintain for the organization to provides products and services to customers.
The Meeting of the Two Universes
DevOps is based on the idea of bridging the gap between the Universe of Development and the Universe of Operations. A gap that traditionally existed since the dawn of IT.
Development is about responding to ever-changing customer needs, exploiting the shift in market demand to the best of your customers’ advantage. Innovation and creativity are at the core of Development.
Operations is about creating and maintaining a reliable, stable, and safe environment for hosting and serving the Products and Services to customers. Stability and Robustness are the core of the Operations.
Development teams like to have the freedom to create prototypes and test the market response with samples they quickly make, to engage the customers quickly into trying them and providing precious feedback that would help the Development teams re-align their work to what seems to be delighting and serving the customers.
Development teams like to “Fail Fast” so they can “Lean Fast” and keep doing that until their Products and Services become a success and resonate greatly with the market.
Operations need robust Products, well developed and tested against all key aspects, from Security to Performance, so they can perform capacity planning, performance scaling, and SLA sustainment.
They essentially despise the assumed “Reckless” behavior of the Development team with a half-measured code that is hastily put together to test the market.
The dramatic difference between the two sides’ agenda, had catered to a conflict between them until the idea of DevOps in joining the two sides to collaborate as one body, in developing responsibly and operationalizing dynamically — creating a win-win stance where both parties are sharing the ownership and accountability for design, development, testing and operating the Products and Services.
When Development teams and Operations Teams meet for the first time, it is usually the first time the Operations Team starts to see the impacts of badly configured platforms and overly tight resourcing on the development process of a Product or Service. It is also usually the first time the Development Team sees the problems with releasing Products and Service without built-in monitoring, testability, and configuration capabilities.
Hands Off! …. from the Hand Offs!
When a work item is passed from one team to another, it causes loss of information in passing the work through miscommunication and a need for ramping up by the next team to be able to continue the work (aka the context-switching which may need preparatory works such as environment setup each time).
This would even be worse if the receiving team is at capacity (have no room at the moment to take in new work because they are all busy finishing their current work).
This would have a direct negative impact on the performance of the DevOps team which can be measured and reflected on the Value Stream Map.
The QA team would send back (Hand Off) the code that has failed to test, to the Developers. This kind of Hand Off may seem inevitable as it is the nature of the work, but the potential for high efficiency here is in finding better ways for integrating the Development with Testing, Automating Testing, Stronger Unit Testing and even TDD (Test Driven Development; a method for building Quality in the development by getting developers to first design the test they expect their code should pass once it is written, and then writing just enough code to pass that test and then refactoring and improving the written code afterward time permitting).
Waiting for Sign-offs and Release approvals when that is the only thing remaining after all coding and testing are done, is another big example of delays through Hand Offs. Teams may be held back for long hours from Release because of one last Sign-off.
Transparency through Kanban
Visualization is key in creating a transparent common vision across the teams. When everyone can have access to see where everything is and how they work is flowing through the pipeline, they can easily contribute to the flow and participate in its improvement.
DevOps teams mostly use the Kanban framework to help them visualize the status of their work and to track and improve their performance and predictability.
Kanban enables the DevOps team to respond to changes as quickly as on a daily basis. It also easily enables them to create multiple swim-lanes (one per class of work / or priority), so work items can travel in parallel at different priority levels.
Kanban is an Agile framework that focuses on creating, running, and improving flow systems for knowledge work.
Kanban does not follow a certain cadence and can accept new work at any point in time, as long as the WIP limits would allow the team members to pull in the new work item into the flow.
Kanban follows a basic set of Principles:
- It encourages the adopting teams to start with what they have and takes it from there by visualizing the work that they are going to do while limiting the Work In Progress (WIP) and avoiding the start of new work items before finishing the ones already in the flow.
- It asks the teams to agree to pursue improvement through evolutionary change, which is a fancy way of asking them to evolve into better teams instead of shutting everything down and try to force one another into better teams.
- It also advocates acts of leadership at every level, which means it asks team members to take charge of the work at hand, own it and feel responsible for it and be ready to make decisions when needed, while it encourages the management to support this level of autonomy in the teams to let them develop the capacity and capability to make decisions that need to happen quickly and close to the location where the work is actually happening while accepting ownership of what they are doing
Kanban follows a small set of Practices that are essential in managing the flow:
- Visualize: Kanban uses its famous Kanban board to visualize the work as it moves from ideation through the steps from one column to another one to completion and deployment. Just remember, as long as your Kanban board can distinguish between these 3 main areas, you are good to go: Pre-Commitment (The area where the work items are brought in for evaluation and assessment and sizing but are NOT yet considered as committed), The Work-In-Progress area (where the work items travel from left to right in its journey for being an idea into becoming a product or service and the Delivered (or Done or Deployed) area, which is the resting place for the work items that are now delivered as part of a release to the customer. (Also remember that there are no rules for how many columns you should have in each area, of what to name them. I would recommend you add as many columns as it makes your teams’ work easier and gives them the granularity, they need to track the steps the work items are going through).
- Limit WIP: Asset for each column — where relevant — to encourage the team to finish what they have started before adding too many other new work items (even if the current work item is blocked by something, the WIP limit would force the needed conversations and activities that are needed to unblock that work item and allow the flow to continue). Also please note that we may want to make sure each swim-lane (which is used to classify work types and their priorities), have their own share of WIP Limit (and are not allowed to go beyond that) while the total number of WIP Limits for the column equals the sum of the WIP Limits for each swim-lane in that column.
- Manage Flow: As the workflows through the Kanban framework, it leads to delivery of value to the customer. The intention here is to incrementally improve the performance by reducing the time it takes the work items to travel across the Kanban board (aka Lead Times), and through that improving the Predictability of teams’ delivery. Naturally, bottlenecks and blockers would hinder our efforts in managing the flow and would need our attention and effort for their removal.
- Making Policies Explicit: Policies are rules and expectations that we add to relevant columns to set prerequisites for entering or leaving that column. Any consideration, criteria, or requirement that is expected for a column is mentioned in its Policy. They should be lean, clear, enough ad visible, and be reasonably expected to be followed by the team. They may include capacity allocation, the definition of done, compliance requirements, data security protocols, and any other rules needed.
- Improve collaboratively, evolve experimentally: Work as a team towards a step-by-step improvement in the team’s performance and delivery quality. Experiment and try different Policies, and WIP Limits and play with capacity allocations across priority levels of your work items to search and find the best configuration that works for you and then make sure to revisit this setting and experiment for the next higher step.
Sins of the Fathers…
As we form up our brand new formations of DevOps teams we should not forget about the Technical Debt (the existing, accumulated set of band-aid fixes and workarounds and deficiencies in our IT that poses an increasing risk to our organization’s existence).
In older enterprises that have been in business for decades, most of the DevOps Team members were not even hired when the Technical Debt has started the downward spiral for the IT teams.
It is as if DevOps is inheriting a Debt, they did not have anything to do with its creation ….
As a best practice, we need to allocate some of our existing capacity to work items that would incrementally carve away the Technical Debt of our organization and will gradually improve it.
It is recommended to allocate up to 20% of our DevOps teams’ capacity to Technical Debt retirement until we completely fix them.
The good news is, as the Technical Debt starts to diminish, it becomes easier and more efficient for the DevOps Teams to push forward with more work items since there will be a continuous drop in Infrastructure problems and environmental issues that would hinder our efforts in retiring the debt.
Written by Arman Kamran, CTO of Prima Recon and Enterprise Transition Expert in Scaled Agile Digital Transformation
