The reality is the seeds of DevOps, which has been maturing for many years, however, in the last couple of years, these seeds have blossomed into maturity to enable a revolution in systems development. Agile methodologies, which always required continuous integration and supported continuous delivery, are now coupled with a growing range of rapidly evolving open source tools that are powering the DevOps movement forward. The open source movement has united developers and is providing development tools that are perfectly tailored to the DevOps approach. Cloud providers such as Amazon can now supply a working environment on demand; it is a developer’s playpen.

Quick moving projects

I don’t want to write this article about the increasingly rich toolkit of DevOps; instead, this article is about how the DevOps revolution is affecting the tester community and how the tester community should embrace DevOps. The role of the tester is changing, but test skills are still required. DevOps places a large dependence on testing, high quality, repeatable automated testing. Applications cannot be released without being tested. So how does the tester fit into the DevOps world, or perhaps it’s better to ask how testing is performed, as the definition of a tester is now possibly changing. Fundamentally, I am a developer at heart and I love quick moving projects with quick delivery, so I’ve embraced agile and the DevOps world. The tools I’m seeing now are what we dreamed of even a few years ago.

But what is the tester in the face of this revolution, and what is the role of the specialist tester as distinct from the test function?

Well, developers have always done unit testing. This doesn’t change in the DevOps world. However, because of the use of automated testing tools, continuous integration and automation servers, unit testing is a lot better in the DevOps world. A good developer will be expected to produce high-quality code and an associated unit test suite as a standard. Unit test level bugs should be reduced dramatically. Also because the unit tests are now tightly coupled to the code, as re-running the automated test suite can carry out regression testing.

Passing a test

The use of techniques such as BDD, (Behavioural Driven Development), TDD, (Test Driven Development) and ATDD, (Acceptance Test Driven Development), in the DevOps world show the importance of testing to the success of DevOps approaches. Continuous testing is the key to the success of DevOps. For example, making use of TDD, a java developer will probably produce a set of Junit tests, which will define the definition of done for the code they are about to produce. They will then produce code and run it against the Junit tests, fix any failures, then keep repeating the process until all test has passed. At this point, the code will be considered done and it can be integrated into the master branch of code. The Junit tests will also integrate so they can be run whenever regression testing is required. So, not only has the code been developed and successfully unit tested, it can be regression tested at any point in time in the future without any additional effort or time.

Unit testing tends to be quite specific to low-level details. BDD and ACDD approaches tend to address the user behaviour level and the user acceptance criteria level. Essentially this means the use of business scenarios that will reflect the behaviour of the function in operation. An example of this would be the use of Cucumber/Gherkin in which the tester can test business scenarios and create an automated suite of tests that can test the code against the acceptance criteria. To write tests in Cucumber/Gherkin requires minimal technical skill and in some cases can be done by the intended users of the system, if requirements were defined in Cucumber/Gherkin then they would produce a testable set of requirements at the outset of any development. However, running and maintaining these tests does require technical skill that end users and manual testers will not possess.

Development language

It is clear that in order to operate as a specialist tester in a DevOps squad, the tester should have good test automation skills and it would be useful if the tester had some knowledge of the development language and the environments being used. The tools mentioned above, as well as the many tools that also support TDD, BDD and ACDD require technical skills to be used to their maximum potential. Automated regression testing eliminates any errors that can be introduced in a manual test phase and it can be run very quickly, within the time span of the actual change.

DevOps has formalised the need for a new role called Software Development Engineer in Test, an SDET. An SDET will be required to perform test automation, creating test infrastructure that can support those tests and potentially support different environments and different languages. This is a role that primarily supports the test function and enables tests to be automated quickly and efficiently by the use of development engineer skills.

Automated test tools and SDETs does not minimise the need for good testing skills, the quality of the product depends more than ever on the ability of the DevOps squad to put in place a good set of tests that will ensure the product meets the expectations of the end users, it’s no good being fast and wrong! Testing tools will not tell you how or what to test. Skilled testers and developers will need to do that.

Mobile marketplace

In a competitive market online, or a mobile marketplace, the need to react quickly to a competitor is essential to success, being first is important but not being too long getting to second place is potentially about survival. The ability to make a change, create a new customer-facing feature or integrate a new supplier, in a short space of time and be able to automatically regression test your platform without any additional effort may enable you to get that change in a few days rather than weeks.

We know that companies, with a good DevOps approach to testing, supported by a well built and managed continuous integration and continuous delivery pipeline, can delegate change to individual DevOps squads who can operate autonomously and manage rapid changes to defined areas of functionality. This can be done because a full regression set can be run whenever a change is implemented. Where a business is able to exploit this is where change is not centrally managed but can be delegated to the appropriate business unit, in this case, the path for a change going from perceived business needs to implement, as software supporting customer growth can be very short.

So are we talking about tester version 3.0, or a reboot of the tester for the DevOps world?

I think we probably are, specialist testers will need to learn automation and appropriate technical skills. Testers can’t be insulated from having to deal with tools such as Docker and cloud environments. However, at the moment, the reality is that very few DevOps projects operate in a totally green field. Most projects have to deal with integrations into existing legacy systems, adhere to corporate and regulatory frameworks, be developed to in-house processes and procedures and be subject to the corporate political influences. DevOps teams may well need to co-exist with existing operational and support teams as well as manage requirements from business change teams. All these external stakeholders will have test requirements that need to be met.

Transforming systems

There is also the need for system integration testing and in large complex systems with legacy components and that will not be easy. Specialist integration testers may well be required. Automation of integration tests in these complex hybrid environments is difficult. New approaches may well need to be found. The need to integrate specialised legacy integration environments to support the end-to-end functionality will place challenges to the automation process.

This is a more complicated environment and the role of the tester will obviously be wider than meeting the development demands of the DevOps team. A strategy often employed is to roll out the DevOps approach to the more dynamic areas of an enterprise, such the online and mobile retail offerings, while the change in other areas of the enterprise is implemented and managed by development and operational teams.

Even in this environment, DevOps is still a powerful enabler, and its power to transform will make it a more powerful influence as companies transform, their IT operations and transform their systems. But it will happen over time, for the moment, nearly all new tester roles will require automation skills as companies will at the very least look to leverage the benefits of test automation even if they do not have a full blown DevOps implementation.

Written by Quality Leader, Wayne Tofteroo

The post DevOps: a developer’s playpen appeared first on DevOps Online North America.

Monitoring the behaviour of software in a live production environment is essentially another form of testing. Faults, software behaviours and consumer interactions with your software can be observed, assessed and appropriate actions taken.

In extreme cases where service has been interrupted, monitoring will provide alerts to developers which will enable a fast assessment of a problem, a timely fix and restoration of service.

Good monitoring will provide alerts as too whether the software is meeting its non-functional KPIs and where it can be improved, also the behaviour of the software in terms of its interaction with its consumers can be monitored.

It is important that the builders of your software understand how the end consumers are interacting with that software. For example, we may have many downloads of a mobile application but few users of the services it provides.

Good monitoring will provide insights into where the mobile application can be improved to increase take-up of your service. Part of the application may have seemed brilliant in the design sessions but not in actual usage with consumers, this can be identified by monitoring user journeys and corrected and hopefully the use of your mobile application based service will improve.

The challenge with monitoring software is what metrics should we collect and how should we interpret the data. machine learning and AI provide the opportunities to dive deep into a vast pool of data points and pull out meaningful insights. It can filter out the noise and identify the data that is relevant to what we are looking for.

Once machine learning and AI have been deployed and set up, it can do this task repeatedly and it can learn continuously as too that data which is both significant and relevant to you.

DevOps and Monitoring

The benefits of the DevOps approach is a clear, fast responsive pipeline delivering IT change that meets the business need when it is needed. A smooth running DevOps operation can deliver immense value to an organisation.

However, creating an efficient DevOps setup is not going to happen overnight. Assembling developers and testers into squads, discrete self-contained and self-managing development units of around 8 to 10 people, and arming them with a complex array of integrating tools with an objective of delivering business value through the timely release of good IT, software will take time.

There will be settling in time required where all the squad members learn how to work together and how to make the best use of the tools and processes. The dynamics in the squad and its behaviours are important as in order for the squad to reach its full potential, all its members must be working closely together. The squad needs to learn how to operate and develop good behaviours. This will happen through good planning but also through trial and error.

However, DevOps is continuously deploying software releases to production. Good DevOps relies on monitoring to be in place production to provide alerts to both new errors and changes in software and consumer behaviour.

Machine Learning and AI can provide insights from the monitoring of the software performance and consumer interaction with the software to highlight issue that may have occurred that can be associated with behaviours in the squad, such as a lack of appropriate testing, being carried out, or not identifying all the required acceptance criteria for a story before it is developed and deployed.

Improvements to DevOps Squad Performance

The goal of all DevOps squads it to continuously improve, both velocity and quality. There are many metrics that can be used to measure the performance of the squad, however, the metric which is most commonly used to deliver insight on the behaviour of the squad is velocity.

The velocity is what can be delivered by the squad in a sprint that is of acceptable quality. Here the squad leader can see what the limiting factors are to achieving quality in the sprint, what new defect stories have been produced, how many user stories were found not to have good enough acceptance criteria and they will have seen how the squad members operated in the sprint. They can then address issues with velocity during sprint retrospectives.

The squad needs to learn what at it can deliver in a sprint and then look to see how it can improve and deliver more in the next sprint. However, DevOps squads can have very low cycle times, sometimes as low as a day or a few hours. Here it is difficult for a Squad Leader to manage the squad and ensure the best behaviours are carried out. The importance of Monitoring and analysis becomes critical in supplying DevOps leaders with the information they require to identify and rectify any issues they may have in the internal processes and behaviours of the squads they manage.

Improvements to DevOps Testing

Measuring the quality of delivered and implemented software is difficult. Testing in the squad is designed to prevent defects from reaching live operation. The most common forms of development and testing in a DevOps squad is TDD, test driven development, BDD, behavioural driven development and ATDD, acceptance test lead development.

TDD is more often used by developers to define what they should develop and BDD and ACDD used by the specialist testers to match the delivery to the acceptance criteria. All these techniques are designed to test that the software will do exactly what is required at the required level of quality. However, testing of this nature essentially verifies the software under predicted conditions and data selected for the test.

The purpose of this type of testing is not to find defects. This means that not all edge cases may not be tested. Good monitoring and AI and Monitoring to generate the analysis will identify those issues when they occur and enable an appropriate assessment to be made of the severity. New User stories can be added to the backlog or fix done immediately.

Monitoring coupled with machine learning and AI will enable important edge cases to be identified and included in automated regression test packs. This will improve the quality of testing and the quality of the delivered product.

Written by Wayne Tofteroo, Head of Test Common Services at DVLA

The post Machine learning to identify correlations between KPI’s and developer behaviours appeared first on DevOps Online North America.

CLICK HERE TO READ ONLINE

Fears that Britain’s largest warship could be vulnerable to cyberattacks

Reporter, Leah Alger, and Editor, Andrew Holt, analyse a surprising case of a new £3.5billion state of the art warship, but with outdated software that left the NHS exposed.

JP Morgan’s ‘Rise of the Machines’

Reporter, Leah Alger, gets an understanding of a thought-provoking topic to be covered by a leading speaker at the Software Testing Conference NORTH in September.

DevOps is a ‘meaningless buzzword’ says report

Reporter, Leah Alger, gets views on a contentious study that says DevOps has ‘zero impact’ on work process.

Thought leadership: 

Using AI to re-imagine software quality
Recognising the power of test automation to deliver business value, Jayashree Natarajan explores the development of AI in software application testing.

Anti-money and laundering challenges

Vice President of Maveric Systems, Krishnakumar Ranganathan, warns that a robust IT approach needs to be embraced to deal with banking Anti-Money Laundering processes.

Testing is broken: How AI and analytics can deliver true test automation

Antony Edwards argues that testing is broken, addressing why organisations struggle with the process.

Device Testing: 

Developing dependable devices

Frederik Van Slycken, Device Security Expert at Intelligent Systems/Altran and contributor to the prpl Foundation, focuses on how to develop a quality, secure IoT device.

Manufacturing Sector:

Manufacturing a better approach to application security

Colin Domoney, Senior Product Innovation Manager at Veracode, looks at what the manufacturing industry is doing well at in terms of appsec and what other industries can learn from them.

Mobile Testing:

Yielding better mobile

Head of Mobile at Future Platforms, Dougie Hoskins, shares his insight on integrating testing into the development process, arguing it can be an effective way to deliver complicated mobile apps.

From Europe to Korea

A case study of globalised crowdtesting.

IoT:

Who’s responsible when IoT gets hacked?

An invasion of privacy and an insecure channel for exploiting the individual are among the accusations levelled at the Internet of Things (IoT), and with some justification, argues Ken Munro, Partner, Pen Test Partners.

Automation:

Digital transformation: driving test trends

Mario Matthee, Head of Research and Development at DVT Global Testing Solutions, says digital transformation is real, but critical mass is still some way off.

Future Vision:

The Test Manager 2.0

Paul Mowat, Technology Delivery Lead Senior Manager at Accenture, gazes into his crystal ball and envisages the future for test managers and the challenges and opportunities ahead.

Smart Cities:

Smart Cities: Lessons Learned

Perves Siddiqui, Vice President of Business Development at Genetec, argues that technology has a big role in enabling collaboration within cities. But we need to ensure we draft the correct policies.

Performance Testing:

Which tool to use in testing

Dasanka Lecic, Test Developer at Levi9 IT Services, looks at the tools to fulfil the job for performance testing.

Vendors to End-Users:

Selling software testing

So you sell software testing services? Ever wondered what your customers are thinking? The answers may surprise you, suggests Johan Steyn.

TDD &BDD:

Efficient practices of development

Elena Moldavskaya, Business Analyst at Intetics, highlights the benefits of the Test Driven development and Behaviour Driven Development approaches.

About TEST Magazine

TEST Magazine was specifically created as a voice for the modern-day software testing and quality assurance professional. Launched in 2008 and distributed to an audience over 14,000, the publication provides news, features and practical, as well as professional advice on all facets of software testing. TEST Magazine is widely regarded as the leading journal for software testing and quality assurance in Europe and beyond.

Interested in writing for TEST Magazine?

Get in touch by emailing the Editor, andrew.holt@31media.co.uk, or the Reporter, leah.alger@31media.co.uk.

Read more from DevOps Online | Follow us on Twitter

The post The July 2017 issue of TEST Magazine is now available to read online appeared first on DevOps Online North America.