It is always an acceptable and/or argumentative fact that the perceptions towards ascertaining the QA/testing needs (encompassing techniques, tools, practices, and strategies etc.) have undergone many ramifications in the recent past, and more predominantly when organisations started to incline more towards adopting today’s agile and DevOps mode of project delivery, which totally changed the very approach for availing the QA and testing services.
One major and notable change was organisations interest in promoting, establishing and investing in TCoE (dedicated centralised testing teams). Although, it is very evident that the agile and DevOps approach was instrumental in expediting the time to market situations, meeting the shorter or frequent release cycles and facilitating continuous delivery. But if organisations are equally looking out to establish a highly standardised QA and testing practice (either for a specific or a complex business need) and aim to build a centralised repository of reusable test assets to deliver quality at optimal cost through optimum utilisation of resources, then TCoE is still the ‘go-to’ solution.
DevOps – the known and unknown
It is very well understood that DevOps is not a methodology, instead, it is a notion to help cut down the barriers between Dev and Ops and purely intended to meet and expedite the needs pertaining to shorter and more frequent release cycles to promote the ‘agile’ mode of delivery with ease. The said notion is achieved through seamless integration and effective collaboration between the assorted teams viz. ‘Dev’ and ‘Ops’ teams, and together, they function as a single entity to accomplish the following activities as a continuous process viz.
- Continuous testing
- Continuous integration
- Continuous delivery
- Continuous monitoring
Although application developers and system engineers bring correct things into the right environments, the actual crux lies with Q&A teams since they need to continuously stay focused to align their test deliverables, and ensure that every minute and/or potential code changes work as intended without actually breaking anything with the application and/or product, regardless of frequent build requests being encouraged.
The reason as to why Q&A teams are considered to be crucial in the DevOps ecosystem is for the fact that they meticulously take care of following key activities continually and quickly in the same sprint (or on-demand basis) for the application under test viz.
- Identify the critical test scenarios
- Develop and automate test cases
- Organise test cases into respective test suites
- Outline the execution order of test cases
- Schedule the execution as part of C.I
- Generate and/or share the test execution results to requisite stakeholders.
From aforementioned, it is understood that the test approach automation adopted by the Q&A team appears to be very systematic with a cohesive test design, which together helps cut down the challenges with test planning and test management. However, the said approach with Q&A in DevOps is definitely not going to be straightforward and result oriented if the applications/products are of intricate nature.
For example, applications and products are designed and developed for BFIS, utility and healthcare (US demographics) and domains are considered to be the most complex. Especially with healthcare domain (US demographics) – it is very exhaustive as it encompasses the most critical pieces of business functions pertaining to
- Payroll/invoice Generation
- Benefits enrollment and disenrollment and their administration
- Payments and remittances
- Processing of claims
Besides, EDI (stands for Electronic Data Interchange) forms the major crux of the healthcare domain (specifically for US demographics, as mandated by HIPPA) since it forms the basis and/or enables the transaction processing through the exchange of sensitive data in an acceptable standardised file format between different heterogeneous systems. Again, it’s not just one file format, but there are several file formats available today each of which is intended to standardise the transactions pertaining to a specific business function.
So, to build such a typical domain centric application or product it not only involves the simulation of a bundle of critical functionalities using contemporary technology stack, it also requires extensive domain knowledge to validate. Especially from a QA standpoint, a robust team encompassing of an exploratory, functional, automation and security testers are very much needed in order to assess the said applications/products thoroughly. However, availability and/or building such QA teams on need basis are one of the major gaps with DevOps.
Contemporary challenges
Following are some of the key challenges that QA teams tend to experience as part of working in the DevOps ecosystem:
Effective Collaboration – apparently, it’s the QA team, which acts as a crucial interface between the product development team and operations. So, it’s imperative that QA teams should be made a part to witness the entire project/product life cycle and its associated discussions, rather than having them involved during the testing phase alone. But this collaboration is of very less possibility of looking at how DevOps operates, and hence there will be potential gaps to both understand and align with the testing needs and expectations.
Test Enablement – it is very critical that QA teams understand the business (and underlying critical functionalities) for which the application/product is built and verified. So, in order to achieve this QA team should team-up and has to work closely with business experts / SME’s to understand how the system supports the business, based on which QA team can ascertain the testing needs and enable the QA services accordingly. But in DevOps, it is seemingly hard to find and/or facilitate this channel of association.
Test Coverage – in DevOps, one cannot guarantee 100% test coverage of the application/product for two fundamental reasons – the rush to deliver the software quickly to meet the release cycles and secondly there are high chances of overlooking from validating critical functionalities due to dynamically changing requirements, which can attribute to defect leakage.
Facilitating Early Testing – one of the major objectives with DevOps is early detection of defects in the development cycle, and this is possible only when testing is planned to begin during the early stages. But it is more likely achieved when there is requisite test documentation (and other supporting test templates) in place to outline, organize and prioritize the user-stories that can be readily tested without any dependencies.
Testing Maturity – regardless of any and every approach with software delivery, the key-differentiating factor from a QA standpoint is the test maturity. Generally, it is one such attribute that cannot be quantified all the time, but it is derived taking certain key factors into consideration such as approach, experience, skill-set, ability to orchestrate and automate etc. So, if the QA teams fail to possess the requisite test maturity, then it is inevitable to see challenges and failures while taking projects through completion.
Centralised testing service (aka TCoE)
The idea behind having TCoE is to establish a highly standardised QA and testing practice besides building reusable test assets and repositories that can deliver quality at optimal cost by optimum utilisation of resources. It helps bring people, process and technology together, and likewise promote the requisite collaboration across teams to improve the effectiveness of testing.
It is a comprehensive structure in itself and is driven by independent and self-sustaining CoE’s outlined below, backed by a strong core team (for governance, facilitation and coordination services), which together act as key accelerators and differentiators to meet any challenging/complex and/or evolving business needs (from QA standpoint) regardless of time to market situations or delivery patterns.
- Functional test competency team
- Domain competency team
- Automation competency team
- Non-functional test (performance, security etc.) team.
The proposition is not all about building big centralised testing teams, or TCoE’s, but to ascertain the possibilities of integrating or embedding already existing centralised testing services into agile or DevOps and maintaining them, to maximise ROI and deliver high customer satisfaction and service standards.
Written by Senior Project Lead QA at Value Labs, Narayana Maruvada
