DevOps goes hand in hand with other well-known practices such as agile and continuous delivery. The need for DevOps arose from the increasing success of agile software development, as that led to organisations wanting to release their software faster and more frequently. DevOps and continuous delivery share a common background in agile methods and lean thinking: small and frequent changes with focused value to the end customer.

To facilitate DevOps we use automated tools for each and every part of the product development stages (code, build, test, package, release, configure and monitoring).

To help the DevOps pipeline in an organisation, many software delivery institutions have now embarked on test automation. With proper test automation tools and practices implemented we are able to test and deliver software with high quality. At the same time, there are a lot of repercussion and nightmares that organisations will face when undertaking a wrong test automation strategy.

Utilise test automation

We all deliver web solutions to our customers. Facebook, Google, Amazon and Microsoft are some of the leading software giants who deliver web-based solutions to billions of users worldwide. These solutions utilise web services and sometimes microservices architectures. Therefore, we embark first on the development of these web services before putting the final UI touches on the system. To facilitate faster delivery we should first automate the testing of such web services rather than waiting for the UI to be integrated into the delivery feature. We should use more web services test automation tools and test frameworks to extensively test web services and microservices architecture. It is a good practice to have more test coverage on web services rather than with a UI based functional test.

The more test issues we see at the foundation level more will be better off in the final stages of testing. These tests should look at testing the web service alone, test web services by integrating with other web services, and test against the contracts established. Web service testing can be automated with tools like RestAssured, Kataloon Studio, Postman and Karate. These are well-known tools in the industry. Web service testing should not limit to functional aspects. It should also be used for performance and security aspects. Tools like JMeter, Load UI and TestingWhiz can be used to test the performance of a web service.

Facilitate continuous delivery

In DevOps, we emphasise on continuous delivery. To facilitate continuous delivery we use continuous integration. Most of us do wrong when using continuous integration for test automation. We only push UI functional automation suite to the continuous integration. Where our automated regression will have to wait until the delivery of new option with UI. We should move away with such practice. The best way is to push our service level automated test into the continuous integration. This allows maximum leverage of our test automation practice. The automated security test at services level should also move into the CI process, which allows us to identify any security risks at earlier stages of development.

Use of the ‘behavioral driven development’ (BDD) approach is another area that also helps application development and automated test development. BDD facilitates creating development code as well as automated code, to what is actually required by the customer, thus eliminating any requirements gaps in delivery and also reworking. We should have a practice where we write automation scripts based on the scenarios (features) accepted by the customers. There are so many add-ons we can utilise for our test automation framework to facilitate BDD. Cucumber for Java and JavaScript, RSpec for Ruby and SpecFlow for .net are some of the add-ons that help this integration.

Codeless testing

We should have a test automation strategy to automate test quickly to keep in phase with the delivery pipeline. Rather than opting for tools that need extensive automation coding, it’s a good practice that we use tools that needs less coding or simply codeless. This helps us to reduce our effort on test automation and speed up the test process. Codeless test automation also helps non-technical quality assurance teams to get involved in test automation with no background in coding knowledge. If they know how to test, they can automate. To put codeless test automation into practice we can use test automation tools like Leap Test, Testim.io or TestCraft.

Machine learning and the use of artificial intelligence (AI) is also an area of focus today, as it will help the teams to bring down testing effort and speed up delivery with high quality. The AI-based test tool should be capable to come up with the test scenarios for a feature, with maximum coverage and design the test automation scripts for it, execute when necessary. There are two great AI automation tools available in the industry. Namely, Appvance IQ and Mabl. With Appvance you can create 1500 test cases in just 5 seconds; a dramatic improvement in testing.

Operating systems

To enhance our DevOps pipeline and to leverage the maximum of test automation we should also utilise the cloud-based testing solutions. These solutions enable us to test against a wider range of operating systems, mobile devices and even web browsers, which we cannot have in-house due to high investment and maintenance cost. Automated testing in such solutions helps to assure that the application delivered works on different platforms and environments. The solutions can utilise our BrowserStack, CrossBrowser Testing, Sauce Labs and Perfecto Cloud.

Another way we can help faster testing, and high-quality delivery is to use tools to help manual testers do more in-depth debugging, helping hunt down more hidden bugs. Tools like Postman and Fiddler helps us to debug web services more in a drill down fashion.

We should also keep in mind that our test automation suite should have the maximum code coverage of the application being tested. There should be more emphasis on code coverage tools like NCover, dotCover, Atlassian Clover and Cobertura. Looking at test automation code coverage helps us to ensure that our test automation code validates most parts of the application.

DevOps success

With DevOps, a test automation engineer’s role expands as they perform more of a developer role. They will have to create automated unit tests to test the developer code and they perform a test drive approach. Automated unit testing is another area where test engineers can embark upon. Therefore, the quality engineering team should work on TestNG, Junit, Mockito and Powermock.

So adding the final points, we need test automation to facilitate DevOps, as it’s a major contributor to DevOps success. So these tweaks that I have highlighted in this article serves a lot of DevOps practitioners around the world to put the ship safely to the sea and start cursing at maximum speeds.

Written by Senior QA Consultant at Virtusa, Kushan Shalindra Amarasiri

The post Useful steps to take when tweaking test automation processes appeared first on DevOps Online North America.

The reality is the seeds of DevOps, which has been maturing for many years, however, in the last couple of years, these seeds have blossomed into maturity to enable a revolution in systems development. Agile methodologies, which always required continuous integration and supported continuous delivery, are now coupled with a growing range of rapidly evolving open source tools that are powering the DevOps movement forward. The open source movement has united developers and is providing development tools that are perfectly tailored to the DevOps approach. Cloud providers such as Amazon can now supply a working environment on demand; it is a developer’s playpen.

Quick moving projects

I don’t want to write this article about the increasingly rich toolkit of DevOps; instead, this article is about how the DevOps revolution is affecting the tester community and how the tester community should embrace DevOps. The role of the tester is changing, but test skills are still required. DevOps places a large dependence on testing, high quality, repeatable automated testing. Applications cannot be released without being tested. So how does the tester fit into the DevOps world, or perhaps it’s better to ask how testing is performed, as the definition of a tester is now possibly changing. Fundamentally, I am a developer at heart and I love quick moving projects with quick delivery, so I’ve embraced agile and the DevOps world. The tools I’m seeing now are what we dreamed of even a few years ago.

But what is the tester in the face of this revolution, and what is the role of the specialist tester as distinct from the test function?

Well, developers have always done unit testing. This doesn’t change in the DevOps world. However, because of the use of automated testing tools, continuous integration and automation servers, unit testing is a lot better in the DevOps world. A good developer will be expected to produce high-quality code and an associated unit test suite as a standard. Unit test level bugs should be reduced dramatically. Also because the unit tests are now tightly coupled to the code, as re-running the automated test suite can carry out regression testing.

Passing a test

The use of techniques such as BDD, (Behavioural Driven Development), TDD, (Test Driven Development) and ATDD, (Acceptance Test Driven Development), in the DevOps world show the importance of testing to the success of DevOps approaches. Continuous testing is the key to the success of DevOps. For example, making use of TDD, a java developer will probably produce a set of Junit tests, which will define the definition of done for the code they are about to produce. They will then produce code and run it against the Junit tests, fix any failures, then keep repeating the process until all test has passed. At this point, the code will be considered done and it can be integrated into the master branch of code. The Junit tests will also integrate so they can be run whenever regression testing is required. So, not only has the code been developed and successfully unit tested, it can be regression tested at any point in time in the future without any additional effort or time.

Unit testing tends to be quite specific to low-level details. BDD and ACDD approaches tend to address the user behaviour level and the user acceptance criteria level. Essentially this means the use of business scenarios that will reflect the behaviour of the function in operation. An example of this would be the use of Cucumber/Gherkin in which the tester can test business scenarios and create an automated suite of tests that can test the code against the acceptance criteria. To write tests in Cucumber/Gherkin requires minimal technical skill and in some cases can be done by the intended users of the system, if requirements were defined in Cucumber/Gherkin then they would produce a testable set of requirements at the outset of any development. However, running and maintaining these tests does require technical skill that end users and manual testers will not possess.

Development language

It is clear that in order to operate as a specialist tester in a DevOps squad, the tester should have good test automation skills and it would be useful if the tester had some knowledge of the development language and the environments being used. The tools mentioned above, as well as the many tools that also support TDD, BDD and ACDD require technical skills to be used to their maximum potential. Automated regression testing eliminates any errors that can be introduced in a manual test phase and it can be run very quickly, within the time span of the actual change.

DevOps has formalised the need for a new role called Software Development Engineer in Test, an SDET. An SDET will be required to perform test automation, creating test infrastructure that can support those tests and potentially support different environments and different languages. This is a role that primarily supports the test function and enables tests to be automated quickly and efficiently by the use of development engineer skills.

Automated test tools and SDETs does not minimise the need for good testing skills, the quality of the product depends more than ever on the ability of the DevOps squad to put in place a good set of tests that will ensure the product meets the expectations of the end users, it’s no good being fast and wrong! Testing tools will not tell you how or what to test. Skilled testers and developers will need to do that.

Mobile marketplace

In a competitive market online, or a mobile marketplace, the need to react quickly to a competitor is essential to success, being first is important but not being too long getting to second place is potentially about survival. The ability to make a change, create a new customer-facing feature or integrate a new supplier, in a short space of time and be able to automatically regression test your platform without any additional effort may enable you to get that change in a few days rather than weeks.

We know that companies, with a good DevOps approach to testing, supported by a well built and managed continuous integration and continuous delivery pipeline, can delegate change to individual DevOps squads who can operate autonomously and manage rapid changes to defined areas of functionality. This can be done because a full regression set can be run whenever a change is implemented. Where a business is able to exploit this is where change is not centrally managed but can be delegated to the appropriate business unit, in this case, the path for a change going from perceived business needs to implement, as software supporting customer growth can be very short.

So are we talking about tester version 3.0, or a reboot of the tester for the DevOps world?

I think we probably are, specialist testers will need to learn automation and appropriate technical skills. Testers can’t be insulated from having to deal with tools such as Docker and cloud environments. However, at the moment, the reality is that very few DevOps projects operate in a totally green field. Most projects have to deal with integrations into existing legacy systems, adhere to corporate and regulatory frameworks, be developed to in-house processes and procedures and be subject to the corporate political influences. DevOps teams may well need to co-exist with existing operational and support teams as well as manage requirements from business change teams. All these external stakeholders will have test requirements that need to be met.

Transforming systems

There is also the need for system integration testing and in large complex systems with legacy components and that will not be easy. Specialist integration testers may well be required. Automation of integration tests in these complex hybrid environments is difficult. New approaches may well need to be found. The need to integrate specialised legacy integration environments to support the end-to-end functionality will place challenges to the automation process.

This is a more complicated environment and the role of the tester will obviously be wider than meeting the development demands of the DevOps team. A strategy often employed is to roll out the DevOps approach to the more dynamic areas of an enterprise, such the online and mobile retail offerings, while the change in other areas of the enterprise is implemented and managed by development and operational teams.

Even in this environment, DevOps is still a powerful enabler, and its power to transform will make it a more powerful influence as companies transform, their IT operations and transform their systems. But it will happen over time, for the moment, nearly all new tester roles will require automation skills as companies will at the very least look to leverage the benefits of test automation even if they do not have a full blown DevOps implementation.

Written by Quality Leader, Wayne Tofteroo

The post DevOps: a developer’s playpen appeared first on DevOps Online North America.