automated testing Archives - DevOps Online North America https://devopsnews.online/tag/automated-testing/ by 31 Media Ltd. Mon, 24 Sep 2018 11:39:28 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.2 Setting up a QA center of excellence https://devopsnews.online/setting-up-a-qa-center-of-excellence/ Thu, 20 Sep 2018 15:40:33 +0000 http://www.devopsonline.co.uk/?p=14062 Dileep Singh Marway, QA Manager, The Economist, explains how he flies the QA flag high by maintaining high-quality standards, at all times, for the weekly newspaper

The post Setting up a QA center of excellence appeared first on DevOps Online North America.

]]>
Dileep Singh Marway, QA Manager, The Economist, explains how he flies the QA flag high by maintaining high-quality standards, at all times, for the weekly newspaper

Marway’s passion is ensuring high quality throughout the application lifecycle and giving back to the testing community by speaking at conferences and contributing to TEST Magazine and other publications.

Like many software testers, Marway has a passion for “shift testing left”, whereby testers are integrated as close to development as possible. He ensures his team has all the support and resources to contribute to this strategy.

“When working for a co-located team across the UK, Belarus and Slovakia, it is key that collaborative communication and teamwork it the centre of all I do,” reveals Marway.

Key tasks

“Setting up a QA centre of excellence means I look for talented engineers in the Midlands and also sell my passion for all things QA.”

Other key tasks he assists with includes:

  • driving the test strategy i.e. deciding when to manually test and its approach
  • detailing tools and technology required for automation testing
  • managing and providing support to his QA team in Birmingham
  • managing and setting clear deliverables to QA partners
  • ensuring quality standards are being set/adhered to across all technology projects. 

For Marway, collaborating and having a shared team vision is key. QA would not be performed well without all team members being passionate about quality. It’s also important they feel they are valued members of the team.

“Driving the test strategy is also important to me. Without a clear strategy, the value of having an efficient QA function would not be clear to the team/other stakeholders,” he continues.

Development skills

“Though I live and breathe the ‘quality first’ approach, it can sometimes occur that stakeholders are more pushed towards the speed of release. It is, therefore, the teams’ job to highlight the risks of going live with minimal QA to the stakeholder.”

Nevertheless, QA can be an underappreciated profession. For instance, if a new game is developed, the development skill may be applauded, without taking into consideration the quality standards that have taken place. Differently, in Marway’s view, a tester is an ‘engineer’, which is a skill not all people have.

“QA professionals are analytical, have excellent attention to detail, and are curious and persuasive. A word, which sums up the profession, is ‘pachydermatous’. It’s important to be thick skinned and always have a quality first approach. On a daily basis, we work with/against developers and stakeholders to ensure that a poor quality output is not released,” Marway adds.

Tools & technology

“In the past, people would begin to test because of being poor at coding, or even over disliking development. My view is that a rounded engineer would understand the code and would help developers to instil quality standards earlier in the lifecycle. This is the main reason why I love my profession, as the role of quality is throughout the whole SDLC and no day is the same.”

Most jobs have their pros and cons, but For Marway, it is a selected privilege to be a QA manager and is something he’s extremely proud of. He explains: “Quality assurance plays a vital role in the success of a company. Given that tools and technology change so quickly in the market, it is a role whereby you need to constantly evolve and pick better strategies to deliver at speed.

“Currently, there are no downsides to my role, I work in an organisation where passion and enthusiasm are never thwarted. I am able to pair up and learn tasks outside of my role, as quality is a consideration for all roles, not just QA.”

Manual vs. automated testing

Nevertheless, like all testers, he has faced challenges, with the biggest being around the initial separation between development and QA, with the “throw over the fence mentality”.

“I am now seeing more of a push to QA being integrated with development, incorporating better unit/integration tests,” Marway adds.

“For me, the bad shift is simply moving from manual to automated testing, automated testing adds value, but only when it is understood and adds business value. QA partners and stakeholders are sometimes more interested in “automating for the sake of automating”, which most certainly is not the right approach.”

Marway also notes that the increase in automation concerns him. According to him: “it will give more physical control to digital systems and make cyber attacks even more dangerous”.

Check out Marway’s tailored QA strategies, as well as his thoughts on “manual testing being alive with new approaches”. 

Written by Leah Alger

The post Setting up a QA center of excellence appeared first on DevOps Online North America.

]]>
Going from DevOps to DevSecOps https://devopsnews.online/going-from-devops-to-devsecops/ Tue, 18 Sep 2018 16:07:18 +0000 http://www.devopsonline.co.uk/?p=14033 As a provider of test leadership for agile and DevOps, an innovator and a believer in change, Wayne Tofteroo believes there are always ways we can improve

The post Going from DevOps to DevSecOps appeared first on DevOps Online North America.

]]>
DevOps is the union between the development tribe and the operations tribe to create an efficient smooth delivery of services. It has revolutionised our approach to development and support. However, it is time to consider the security tribe and to move security over to the left and make it core to what we do in DevOps: we need to move to DevSecOps.

Data is now more precious that gold, diamonds or shares in Apple. There are relentless and irresistible groups that are using increasingly sophisticated tools and techniques to penetrate defences and harvest data. The threat can come from state actors, criminal organisations, competitors or lone-wolf hackers determined to gain kudos and peer recognition. It seems like every week there is another major data breach, or we are sent a new credit card as the credentials have been compromised.

Attackers are creative, persistent and varied in their modes of attack. A security policy needs to take into account the likelihood of a determined attack being carried out across several different vectors simultaneously. They can attack through all technical channels as well as directly attempt to create social and behavioural attacks to gain access. A determined attacker can be patient, gain limited access and use that access to compile enough information to fully breach a system and steal information.

Traditionally security testing is something that happens late-stage in the development process. With DevOps we want to test and deploy continuously. Can we do this securely?

The threats

As data becomes more valuable and hacking a computer system becomes a safer option than robbing a bank. Organised crime will try and make a viable income stream from cyber theft. This means that more time and money will be invested by larger, better-funded actors. This means that the threats will increase in sophistication and diversity and at best we can only stay one step ahead. National states are increasingly turning to cyber weaponry to achieve their state goals. One outcome of this is the inevitable trickle-down of this cyber weapon technology into the hands of cyber-criminal groups who can modify it to achieve their goals, changing the payload to enable the stealthy delivery of malware.

I think we have to move away from the concept of security testing as an activity and start to consider security as an end-to-end process that is holistic across the enterprise and deeply embedded in everything we do.

Security by design

Security needs to be built into everything we engineer, for that to happen smoothly, security requirements need to be embedded into the design process in the same way that functional requirements are. This includes the infrastructure, network and applications. Guidelines for developers need to be produced that inform on secure engineering practices. When developers come into a DevOps organisation, the tools and frameworks should be in place to enable them to understand the development practices and standards to use. Known vulnerabilities need to be explicitly avoided, they should be included in design guidelines and developers’ coding standards. The key concept here is that we are in control of managing security risks from the outset and that security policymakers and developers are making informed decisions about security from the outset.

Scanning, automated testing, peer reviews

We need to develop and test continuously. We can integrate scanning tools into the DevOps pipeline and run these in the same way we continuously test functional code. Cloud infrastructure tools such as Microsoft Azure Advisor and third-party cloud scanning tools can be run as well as application scanners, such as OWASP ZAP, that will look for vulnerabilities in application code, should be considered in development squads and teams. It isn’t a total solution but will identify where known types of vulnerabilities have crept into the development process. Vulnerabilities can then be dealt with like any other defects in the code. They can then be prioritised.

Security tests can be automated and run as part of the DevOps pipeline in the same way as functional tests are. Tools such as BDD-Security or Gauntlet.io, can provide the automation test frameworks needed to run security test alongside functional tests.

Regular peer reviews and open discussions on security will help keep team members focused on security and emphasise the importance of keeping to build patterns. These techniques are not so much as to enforce the design guidelines but to keep everyone focused on the goal of building a secure application.

Penetration testing

Penetration testing, or ethical hacking, give security testers the opportunity to look for vulnerabilities in the solution as a whole. Penetration testers take on the role of hackers and perform simulated attacks on the system. However, while it is a necessary component in your cyber security arsenal, it is a manual process and cannot be integrated into the traditional DevOps pipeline. The purpose of penetration testing is to prove the design and build of the security requirements are sufficient to prevent malicious attacks. A policy decision needs to be made on where and when to use this technique. In the world of continuous deployment, our applications are changing, potentially many times a day. We need to determine what type of release we will use this technique on. Perhaps a policy of regular cycles of penetration testing might be an option.

Monitoring & analysis

The answer to the question of ‘what should be monitored?’ should be ‘everything that can be monitored!’. All system events should be monitored and analysed, including all system and firewall activity. I actually think this is now the most important form of defence that we now have. We have to accept that, despite all our security policies and defences, an attacker will find a way through. A way of identifying an attack might be in identifying a change in system behaviour or firewall activity.

An example could be where file-less malware has gotten in and is monitoring activity internally and send intelligence to an external entity in preparation for a more mature and dedicated attack. File-less malware attacks have risen by 94% in the first half of 2018, (SentinalOne), and are difficult for legacy anti-virus tools to identify. Other attack types are also increasing and the ability to prevent an attack from penetrating your defences is being challenged every minute of the day by new and unknown attackers.

Having a good picture of how your system operates when uninfected and monitoring for any deviations seems to be a sensible idea and using analysis tools, such as AI, to rapidly identify any unexpected deviation from the norm can only be a good thing.

Analysis tools can provide alerts and automatically generate issues that can be fed to a team’s backlog for prioritisation or immediate fix. AI tools can learn and adjust to new threats as they evolve.

Social & behavioural security

Although the above are means of creating a DevSecOps pipeline, the problem of securing the enterprise is not just technical, it is a ‘people problem’ as well. As we harden up our cyber-defences and become more security conscious, our people and processes will become relatively easier options to attackers.

An increasing number of attackers are collecting information on the internal workings and processes of an enterprise with the intention of developing a full-blown attack – where the attacker is making use of that internal knowledge of the organisation and exploiting the internal biases and cognitive vulnerabilities of the staff. This type of threat is now known as ‘advanced persistent threat’ and it is growing.

The only true defence against the infiltration of your internal communication processes is to increase the use of secondary validation techniques, a variation on secondary authentication. It is now no longer safe to authorise the change of payment details for a supplier on the strength of an e-mail from a known contact, the supplier’s e-mail system may have been hijacked and the mail is the result of intelligence gathering malware and a patient attacker developing the attack over time. A secondary form of authorisation needs to happen, such as a phone call, but not to a number on the e-mail!

These processes need to be thought through and prepared for as part of the attack will be to increase pressure on the recipient by saying that it is an urgent request, or by playing on a friendship or goodwill. Persuasion techniques are well known and will be integrated into the attack when it is played out.

Summary

Good security is not just about creating a DevSecOps pipeline. This will be a key component of it. It is a combination of many things that need to be thought through and understood clearly. A clear security policy needs to be in place and security needs to be an integral part of the DNA and thinking of the organisation. It isn’t just technology and it isn’t just people. The system we are protecting is the people and the technology interacting together.

Written by Quality Leader, Wayne Tofteroo

The post Going from DevOps to DevSecOps appeared first on DevOps Online North America.

]]>