Testing mobile apps in the cloud were already on the rise before 2020, but the pandemic turned it into a must. Physical devices on which to test — whether smartphones or tablets — were left stranded in the office and getting new ones to remote teams suddenly based at home was difficult and expensive.

The situation put the spotlight on the limitations of on-premise testing across various devices and permutations. Practically speaking, testing mobile and web apps on physical devices means having someone physically present to maintain them, such as making sure that they are switched on and that the latest operating system update has been implemented. Additionally, for many organisations, these devices interact with backend security systems and they require proper access to these environments.

At the same time, keeping up with the mobile market has become even more difficult, due to its fragmented nature, particularly the diversity of Android, which covers Samsung, Google, Huawei, LG, and more. Android stock operating systems are created into those vendors’ own unique, customized operating systems, further adding to the complications of testing.

While having fewer variants, Apple also brings complexity with the introduction of iPadOS,  deployed in parallel to iOS. Apple also introduced a new application concept called AppClips, which expands the boundaries of mobile testing toward web and sensor-based app deployments.

Furthermore, browsers also introduce fragmentation. Progressive web apps (PWAs) are becoming a new normal for web app developers trying to expand into the mobile landscape. PWAs close technological gaps with sensor integration, storage, access to contact lists, mobile push notifications, and more.

As well as the sheer logistics of addressing all the permutations, testing on physical premises is costly, not least due to the number of devices to acquire. Devices may also be hard to acquire, particularly at pre or at-launch stage.

Cloud-testing benefits 

Running mobile devices in the cloud overcomes the limitations of on-premise testing. Apart from the obvious benefit of being able to support remote operations, clouds also make the whole process much faster, without the need to create physical lab environments. Plus, by moving to the cloud, it is much easier to get immediate access to the latest releases, rather than wait in turn for availability.

Far more parallel executions can be carried out, with scaled-up volumes of tests being concurrently run up or down. For example, teams can run 1000 test cases on 10 mobile devices or on 1,000 mobile devices, depending on what is required within each development sprint. Without having to set up and maintain physical devices, time can be better spent on other tasks.

Creating digital environments in the cloud also allows exploration of far more real user scenarios across multiple variables, such as network conditions, 4G or 5G, Wi-Fi, screen sizes and resolutions, competing background applications, pop-ups, languages, security alerts and push notifications.

Better for team culture  

Since the cloud is technology-agnostic, teams can choose their preferred tools with which to synchronise testing. For example, Jenkins, Bitrise, Android Studio, IntelliJ, and Eclipse, as well as test automation frameworks, including Appium, Espresso, Selenium, and XCUITest all integrate with clouds.

Improved collaboration is possible because everyone can view the same ‘pane-of-glass.’ Whether a developer, test or QA professional, or a product manager, all contributors can view and review testing activities in the cloud. In turn, this universal visibility supports better Agile and DevOps adoption.

Regardless of whether testing manually, exploring test automation for the first time, or expanding on existing test automation strategies, the cloud has a role to play. While most teams are progressively moving toward more automation, manual testing will always be used by some organisations, or at least to an extent. The cloud can support various manual testing use cases, such as scanning a QR code or checking out performance depending on different locations around the world.

For teams keen to speed up the transition to automated testing, the cloud is an excellent foundation from which to build. Tests can be run on a variety of phones, browsers, and versions, and more to the point, they can be run in parallel. For organisations that have already adopted test automation, the cloud gives them a fast track to even greater scalability.

Best practices 

Of course, like any theory, successful execution is what counts, and there are a few aspects to consider for cloud-based testing.

The first is security, compliance, and privacy: are data centers located on each continent in order to satisfy regional regulations? For global coverage and high-performance testing, the clouds need to be distributed across continents. Cloud deployments and their security options are critical for governed and productive development and testing processes. Being able to support SSO, VPN, and gain access to the backend services is key for end-to-end testing. Also, staff working remotely should be equipped with VPN access so that they have the same robust security at home as they would in the office.

One of the key challenges prior to moving to a cloud solution is keeping up with the latest platform support and availability. Each month a new browser version is released and it needs to be supported (including beta). Every few months, new mobile devices and operating systems are released. These must be immediately supported prior to end users experiencing any issues.

Also consider the tools with which cloud testing can integrate, such as the preferred CI/CD toolchain. How open is the integration approach? Make sure that there is support for multiple testing types, including functional, accessibility, performance, exploratory, and more. Plus, keep in mind the ability to scale up and down as each project requires.

Think about choosing a cloud solution with cross-team collaboration and administration tools. These have the ability to manage multiple teams and allocate devices or browsers for testing by roles, tasks, and so on. This is an important feature for better utilisation and productivity. Additionally, repository management centralizes assets for the entire team by uploading a new binary for testing into a centralized repository where all team members pick it up for testing.

Certain widgets and testing features can also speed up testing. Look for clouds that enable typing text, rotating devices, connecting from a mobile device to IDE, and injecting image/location/audio. Finally, it is important not to forget about reporting and visibility, with the option to monitor and analyse all test executions.

Cloud-based testing is evolving and maturing fast, with proven best practices and a wide variety of options to suit each organisation’s needs. Now is an ideal time to investigate or expand testing in the cloud.

Article written by Eran Kinsbruner, Chief Evangelist, Perfecto