QA InfoTech had recently done a webinar on smart devices test automation. Recording from the session, including test automation demonstrations are available here
Given the importance of this topic, I wanted to briefly summarize the webinar’s content in this blog, to enable a quick read. At a very high level, by smart devices we refer to devices that enable voice, video communication, have access to the internet and support varied types of applications to enable end user computing.
Even before one starts thinking about smart devices application test automation, let’s understand the overall landscape or possibilities to consider in building a comprehensive testing strategy covering both manual and automated techniques. We’ll look at this also from both the software and the hardware angles. On the software side, we have the underlying platform and the supported applications at play. The platform ranges across closed and open source options all the way from Ios, Windows Mobile to Android, Meego and the like. As for applications that are supported on these platforms, you have native built in applications as part of the vendor’s repository, web applications that are extended to work on mobile platforms, as well as hybrid applications. Depending on what kind of application needs to be tested, the strategy needs to be customized.
On the hardware side, the form factor options are plentiful. You have smart devices built as flips, touch screens, sliders, bars to name a few of them. Although these don’t necessarily impact testing directly, what is important is to understand the associated input type to see whether the instructions are fed in through a keyboard, using a touch screen and spoken out. This is an important variable to keep in mind in prod3izing your test matrix.
If you were to look at the testing strategies from a manual and automation standpoints, a large portion of the test efforts still happen to be manual, because compatibility testing is what is typically primarily done on devices; a large portion of the functionality would have already been tested for on a base platform. Since having all devices in-house is often expensive and does not provide a good return on investment in terms of usage, companies resort to alternate strategies such as leveraging devices for testing on the cloud, using crowd sourced testers who have a broad set of devices, using device emulators etc.
When it comes to test automation for mobile devices the market is still evolving. Test automation tools for non-mobile scenarios may not always work out of the box for mobile applications. They may need additional plug-ins, add-ons or may be a separate tool altogether for the requirement at hand. One needs to keep in mind the application that is to be automated (native, web, hybrid) and also the underlying operating system (iOS, Android, Windows etc.) before deciding what tools to use. For some tools, the source code access may be needed while others can be designed and implemented based on the available executables.
Some of the more popular tools include: See Test, QTP with M-eux Test, Selenium, Sikuli with Screencast driven by image based UI automation, MonkeyTalk, Robotium. One needs to consider individual tool’s requirements in terms of access to source code, whether the automation is image based or not (if it is, it is expensive to maintain) and the other factors mentioned in the post above in deciding which tool to use. The bottom line is that test automation for smart device applications is feasible although evolving – understand the constraints you operate within and start in small batches where return on investment will be high to devise a successful test automation strategy.