This post is frequently updated. Last updated: April 1, 2021
Although the demand for desktop app testing is not growing as fast as mobile and web app testing, it’s still a crucial day-to-day duty of many testers, especially those who work on sophisticated legacy systems in complex enterprise environments. Ignoring desktop app automation is simply a luxury that they can’t afford.

Desktop applications are computer programs that must be installed, configured and executed on a desktop operating system such as Windows, Linux and macOS. End-users need a desktop computer or laptop with sufficient hardware resources to access the functionalities of these apps.

Common technologies to build desktop apps include Java (Swing, AWT, SWT, JavaFX), .NET (UWP, WPF, WinForms, VB.NET), Windows API (win32), GDI, MFC, Visual Basic, Visual C++, PowerBuilder, SAPGUI, Oracle Forms, Telerik, DevExpress, Infragistics, Xlib, Cocoa, etc.

This article lists out the best Automation tools for automating functional testing of desktop apps. If you’re looking for an Automation tool, you’ll find this list convenient to explore, evaluate, compare and eventually adopt the best tool for your Automation needs.

1. WinAppDriver
2. Winium
3. TestArchitect
4. Ranorex
5. Tricentis Tosca
6. TestComplete
7. Micro Focus UFT (QTP)
8. SikuliX
9. Robot Framework
10. AutoIt
11. White Framework
12. Conformiq

Standard Criteria

When assessing the pros and cons of a specific tool, I used the following criteria.

#1. WinAppDriver

WinAppDriver is built on top of the WebDriver protocol – the gold standard for web & mobile app testing. You can use WinAppDriver to drive any Windows apps such as classic Windows controls (Win32), WPF, WinForms, and UWP using a wide variety of language bindings (C#, Java, Python, Ruby), and test runners (MSTest, JUnit, Nunit, etc.).

If you’re only interested in automating Windows apps with WebDriver, and you know how to code, you should go with WinAppDriver. If you’re not confident with coding, adopting WinAppDriver may not be the best option for you. If you’re not sure how to build a well-structured WebDriver-based framework, check out this post for ideas.

#2. Winium

Winium is an open-source Test Automation framework for WinForms and WPF app testing. Just like WinAppDriver, it’s based on WebDriver protocol. However, the tool is considerably weaker than WinAppDriver in several areas.

For open-source projects, active maintenance is key. Winium has not been actively maintained since 2016 so if you go with it, you have to fix bugs or add new improvements on your own. Your team needs extensive technical knowledge and skills. Ultimately, Winium is a less desirable choice than WinApp driver.

#3. TestArchitect

TestArchitect is a Keyword-Driven Testing tool. Testers with modest programming skills can learn quickly thanks to an extensive keyword library. TestArchitect also features a modern Keyword-Driven Testing method called Action-Based Testing. TestArchitect appeared in Forrester Research reports in recent years.

TestArchitect is relatively stronger than open-source solutions in terms of usability, width, and depth of support. If you don’t want to build a framework from scratch, you should go with this well-rounded framework. Besides, the tool is well maintained by a professional engineering team, unlike Winium. The caveat? You need a budget for it (perpetual or annual). Luckily, the tool does offer a Freemium version with a limited number of test cases.

#4. Ranorex

Ranorex is a Test Automation tool that has been in the market for a long time. They offer proprietary Automation technologies to automate a wide range of desktop, web, and mobile apps. Ranorex also appeared in Forrester Research reports in recent years.

Ranorex’s presence in the Japanese market is strong thanks to their Japanese localization and reliable record & playback. For other markets, Ranorex is not that strong. Invest in Ranorex if you love record & playback as the primary method of test development. 

#5. Tricentis Tosca

Tricentis Tosca provides wide support for virtually all test activities. They offer customers a Continuous Testing platform that covers test design, UI testing, API testing, service virtualization, and test data management. With the recent acquisition of QASymphony (qTest), they further expand their scope to test management.

Tricentis Tosca is very friendly to manual testers who want to transition to test automation engineers. The company claims that given 3 weeks, they can train 85 out of 100 manual testers to become automation engineers using Tosca. On the contrary, only 15 out of 100 manual testers became Selenium engineers given 3 months of training (yes it’s months, not weeks). However, albeit easier to adopt at first, Tricentis Tosca will gradually become more complex and difficult to use in the long run due to the weight of Model-Based Testing.

#6. SmartBear TestComplete

SmartBear TestComplete has been on the Automation market since 1999 so it’s obviously one of the veterans. TestComplete (TC) offers a comprehensive solution that covers desktop, web, and mobile. They’ve recently added a hybrid UI matching method combining visual recognition and object recognition.

TestComplete is indeed one of the best tools on this list. Consider checking it out.

#7. Micro Focus UFT (formerly HP QTP)

UFT/QTP is one of the oldest tools that still survives in the Test Automation market since the very early days. UFT is a powerful UI testing tool. It allows you to test a wide range of platforms including desktop, web, web services, SAP, and mobile. It also integrates seamlessly with HP Quality Center.

If you already have HP QC as your test management system, UFT is your natural choice for Test Automation.

#8. SikuliX

SikuliX is an open-source tool that automates anything you see on screen using image recognition and OCR. The name actually means “God’s Eye” in Huichol Indian culture. Check it out here.

SikuliX is suitable for small projects since it was designed to automate various mundane tasks on Windows. However, once you scale up, you’ll find unsolvable problems due to the nature of the image recognition technique. Additionally, image recognition is way slower than object recognition.

#9. Robot Framework

Robot Framework is another open-source tool that is keyword-driven. The robot is flexible since you could use it with many desktop Automation libraries such as SwingLibrary (Java Swing GUI), SikuliLibrary (SikuliX), SapGuiLibrary, JavaFXLibrary (JavaFX apps), EclipseLibrary (Eclipse RCP/SWT), and AutoItLibrary (AutoIt).

Robot Framework is an excellent alternative to expensive commercial tools. If your budget is tight, consider adopting Robot FW. However, beware of the not-so-obvious investment in learning and maintaining the framework in the long run.

#10. AutoIt

AutoIt is a freeware designed to automate Windows GUI. Although the initial purpose of AutoIt was to “roll out” and configure thousands of PCs, you can use AutoIt for Test Automation, too. Fun fact: AutoIt became popular in part thanks to Selenium’s inability to interact with Windows dialogs that you manually use to upload files to a web page.

AutoIt works for some simple Automation scenarios. If you want a free and lightweight solution to get through simple Automation challenges, use AutoIt. Also note that if you like Selenium, you ought to use AutoIt to work around pop-up dialogs.

#11. White Framework

White framework helps you automate rich client applications written by Win32, WinForms, WPF, Silverlight and JavaSWT. It’s based on .NET and depends on Microsoft’s UIAutomation (UIA) to drive desktop apps.

White is not a strong contender on this list due to its lack of capabilities and narrow target Automation platforms. Try to avoid using White if you want a maintainable tool that can help scale up your test project. That said, small quick and dirty projects can still leverage White thanks to its free cost.

#12. Conformiq

Conformiq is not strictly an Automation tool. Its core offerings (Creator and Transformer) are tools that help you model the business flows of the app under test, generate tests and transform those tests to executable scripts. Conformiq totally relies on third-party Automation libraries such as QTP/UFT for desktop testing and Selenium for web testing.

Conformiq works really well for test teams with more business expertise and less technical. It helps you visualize the test flows as a whole through a model so you don’t miss test cases and bugs. However, having to purchase additional UFT licenses is a big entry barrier.

Conclusion

This list was composed as of early 2019 and has been continuously updated ever since. The tool market for Automation Testing is very competitive. So you can expect radical changes and innovative features being introduced every day. If you spot some outdated information, please leave a comment. I’ll fix it ASAP!

Have you seen better desktop app testing tools not listed above? Comment the tool names and why you think they are the best below. I’ll add them as soon as possible.

Disclaimer: This article is not endorsed by, directly affiliated with, maintained, authorized, or sponsored by any of the companies mentioned in this article (Microsoft, Ranorex, Tricentis, SmartBear LLC, HP, Micro Focus, and the listed open-source project teams). All product and company names are the registered trademarks of their original owners. The use of any trade name or trademark is for identification and reference purposes only and does not imply any association with the trademark holder or their product brand.

Share:

Thuc Nguyen

Thuc Nguyen has been leading the product teams at LogiGear in delivering quality test automation solutions to LogiGear's customers and services clients. Thuc has a great passion for helping organizations transform their Test Automation, Continuous Delivery and DevOps practices as well as empowering testers of all technical levels to thrive in complex enterprise environments.