In the world of independent software verification and validation, nothing ever happens in isolation and, at times, it is all too easy to forget the critical ‘independent’ viewpoint, with the potential result that corporate reputations can be seriously tarnished.
Test effort is cumulative. But it is not until the arrival of a period of sustained business interaction that the diligence of accumulated verification ‘fact’ has the chance to dovetail with the creativity of commercial validation ‘opinion’.
Such a window affords the opportunity to combine the discipline of crafted test case execution with exploratory testing imagination.
In a waterfall-world, that occasion is usually referred to as the UAT test phase.
In an agile world, where the desire is to obtain the broadest business approval across a related number of sprints, a label of UAT Hardening is most often used. In such a way, the prospect of paying more than cursory lip service to individual archipelago land masses is present – an occasion to really join up the proverbial dots, so to speak.
Howsoever named, this sustained business user interaction is the first time these users will have had sight of their new functionality and also time to give it a good kicking. Here is where the most convincing evidence accrues that illustrates whether the right product is being built and if it is being built the right way. In the eyes of many, this is the most critical period of testing in an entire project.
Test validation activity
Left to their own separate devices, neither test verification effort nor test validation activity is likely to be as effective as when a practical, well-resourced and agreed project plan bonds them together. To realise such an objective, of course, requires early and on-going stakeholder communication to bring about that goal.
It also requires training that is timely, well organised and enjoyable. Excellence in this realm is seldom achieved by uninspired individuals who merely regurgitate slides they themselves saw for the first time a few weeks previously.
By way of illustration, one of the most effective and enjoyable training experiences that I have had the pleasure of witnessing was the education associated with a worldwide series of training workshops for a new database offering. The fellow who designed and authored that courseware did a remarkable job in creating original material that including cardboard cut-outs which, when folded origami-style (his wife is a Japanese lady), greatly assisted understanding of challenging technical topics. He had allegedly spent much time sitting on the desks of his development and sales colleagues, not going away until he obtained the information he required. Rumours also circulated that he occasionally brought his cat into work for some weekend company and inspiration and, as further testament to his character, it turned out those tales were true and the cat was called Garfield!
But, it was this same character who was a natural teacher and one that knew how to create magic in the classroom, so much so, he was earmarked for a seven-week trip to Toronto, San Francisco, Sydney, Wellington, Singapore, Amsterdam and London to deliver the material he shaped to global technical teams. Furthermore, so well was that training received, that he was persuaded to visit those locations again to deliver the same innovative material to the sales teams there. With business-class travel, naturally.
The message here is to seek out the best-of-the-best in this arena since training is such a key enabler of every test verification and validation activity.
Collaborative teamwork
Returning now to the wider world of testing and to the subject of collaborative teamwork in particular, where there is a very real and present danger.
These days the majority of senior testing vacancies emphasise the need for collaboration above pretty much everything else. Collaboration with the development teams, collaboration with the service management function, collaboration with business users and their line management and collaboration with the technical BAU functions. Plus, collaboration with other project teams to ensure there are no nasty surprises hiding in anyone else’s woodwork, either.
Most of the time, a collaborative mentality is good testing practice. Questions are asked and answered. Problems are discussed and resolved, practical test artefacts get created and approved. Ergo, confidence grows in the ability of the entire team to create verification and validation excellence.
Every now and then, however, something rather important called ‘test independence’ gets forgotten.
In the same way that the overriding responsibility of every auditor is to form an independent opinion of the accuracy of an organisation’s accounts for its shareholders’ guidance, and every physician who is charged with upholding the Hippocratic Oath; the professional test manager is mandated with providing an independent assessment of the status of testing on behalf of project stakeholders.
Communications & migration
One might well ask, ‘if not him or her, then who?’
Consider the hypothetical situation where a major financial institution desires to schedule significant migration activity over a particular weekend. Relevant communications go out, together with notification that certain commonly-used customer routes to affected data would be limited during the migration window. All well and good, so far.
But, let us further assume that when the anointed weekend arrives, pre-migration test activity is substantially incomplete and that this status is known to senior management.
However, the go-ahead for migration is given with the result that a multitude of serious data errors occur when a subsequent access is attempted by customers using their usual access routes.
Questions would, therefore, have to be asked, at levels up to and including high places, to establish whether the in-house test function and their out-house advisors had been collaborating with other project team members to such a degree that they could no longer see the proverbial wood for the trees.
This is where professional ethics and an ability to sleep nights have to play their role in forming and delivering a complete, truthful and independent picture.
In such a situation, one would also trust that those who not only defined but also agreed the test approach would face some fairly blunt cross-examination.
There are times when a risk-based test approach is not the correct one. When a certain amount of time is absolutely required for a test team to collect an appropriate amount of functional and non-functional test evidence, then that time must be allowed if a respected commercial reputation is not to be seriously tarnished.
Which is why an independent software verification and validation voice is so critical.
The ides of over-collaboration
Earlier, we touched on good testing practice. In cases such as the hypothetical situation described, perhaps the best advice is for every test manager to be aware of the Ides of Over-Collaboration. There comes a time when collaboration must be trumped by the principle of independence so that any subsequent charge of ‘conflict of interest’ can be avoided. Such a view may have its detractors, but there is mounting evidence the attitude of a sensible Chief Information Officer is aligned with that of an independent test function. In a recent CIO article, directional wisdom was expressed as not only the ability to get colleagues rowing in the same direction in times of disagreement but also to be fearless in explaining the rationale, obtaining commitment and making the tough calls in finding a path through. To talk straight, be honest and get comfortable dealing with conflict was the other valuable advice given. Well said, and thank you, Jo Abernathy.
Before closing, there is one other aspect to collaboration that is so fundamental that it must not be subject to the principle of independence outlined above.
A while back, in rural Hampshire, a situation occurred during a large programme when an onshore developer fell and sprained her ankle. Painful, but it happens. Here, it was the test manager (the leader of the ‘traditional enemy’) who rushed the patient to the hospital. Not because he had to. But because the human aspect of any collaborative situation must trump even an independent point of view. For the record, the developer recovered well but had to put both feet up for a day or two.
That one exception aside, every test manager worth their salt needs to listen to the voice in their head that reminds him or her that their primary obligation is to deliver a timely and independent test verdict for the benefit of all stakeholders – and stand by it.
In a number of his books, Sir Terry Pratchett poses the question quis custodiet ipsos custodes? (who watches the watchmen?) At some point, the test profession is going to have to give that question some serious thought and come up with a robust answer.
Written by Ivor Kelly, Head of Testing & UAT Consultant
