The coded UI framework in Visual Studio has record and playback functionality for test case creation as well as the flexibility to write tests in code. Incorporating a coded UI test platform in Visual Studio helped expand the ability for team collaboration but the burden of maintaining coded UI tests creates limitations for continuous builds and large-scale testing needs.
Coded UI tests are fragile by nature and are easily broken by frequent UI changes, and are also slower to run than unit tests or mocked tests. In many cases, developers have to determine at what point and how much testing built on coded UI is appropriate. Testing early in the development cycle when the application’s user interface is changing frequently can result in a high test maintenance cost, while waiting until later in the cycle can result increased development time.
The fragile nature of coded UI tests prevent reaching critical mass for test automation. After automating many tests, teams find themselves spending more time maintaining old tests than creating new ones.
TestArchitect for Visual Studio is a keyword authoring platform extension designed specifically to enhance coded UI test automation in Visual Studio 2012 and make it possible to increase test coverage and execute large-scale test automation. As an extension, it fully integrates with Visual Studio and allows keyword-driven tests to be created for all development platforms that are supported by coded UI in Visual Studio 2012, native windows WPF applications and Windows Internet Explorer browsers.
By using TestArchitect for test authoring, test creation is faster and more importantly, tests are long-term maintainable and highly reusable. The improvement in test creation and automation enhances ALM within the Visual Studio environment.
“LogiGear’s TestArchitect being integrated into Visual Studio 2012 is a shining example of how automated, scalable testing within a development environment results in faster deployment of high-quality software by enabling test engineers and developers to expedite comprehensive testing,” Tom Lindeman, Director of the Visual Studio Industry Partner program at Microsoft, Inc.
Modern keyword-based testing
The TestArchitect test authoring platform gets its roots from the Action Based Testing (ABT) approach pioneered by Hans Buwalda over a decade ago to address the challenges of testing increasingly complex software.
ABT uses actions (defined by keywords) that serve as building blocks for test cases. Actions can be used to create test cases, and combined to create more complex actions. Both actions and test cases are reusable so test development becomes faster with every test.
While there are similarities in the technical implementation of keywords, ABT embodies a test design methodology and a production process that focus on the three essential pillars:
- Test system design: Effective break down of the tests.
- Test module design: Right approach per test module.
- Test case design: Right level of test abstraction.
This framework makes it possible to create tests that are far less fragile when applications change. The benefits are long-term maintainability and high reusability of tests.
A significant advantage of TestArchitect’s modular building block approach to test design is that actions, interfaces and test lines are all separate. When requirements change or a new platform is added, only minimal modifications are needed to maintain the tests. This also makes it possible to design and write tests before functionality is complete. For teams that have embraced Agile practices or operate in rapid development environments, the ability to create tests early adds efficiency.
Modular test organization with TestArchitect for Visual StudioTest cases are created from a series of keywords with arguments. The automation focuses not on automating test cases, but on automating the keywords. Since there will be fewer keywords than test cases, and keyword implementations tend to be shorter than scripted test case implementations, the automation effort is faster and much more manageable.
In a well-designed keyword based test suite, only a limited number of keywords have to be maintained when an application changes. Apart from the more manageable and maintainable automation, a keyword test is more readable and becomes self-documenting compared to a verbose format where test steps are described in natural language.
All test development and execution in TestArchitect is based on test modules. Test modules are work files—very specialized spreadsheets—where tests are created. Test modules contain the following elements:
- Objectives – A statement of what must be tested in the software under test.
- Test cases – One or more actions followed by one or more checks, usually simulating end-user interaction with the software under test.
- Actions- A series of operations and checks.
Test modules act as containers for a group of test cases with a similar, narrowly defined scope. Proper organization of test modules is an essential factor in realizing efficiency gains.
Test modules should be well defined and differentiated from each other. Test cases within a test module can have dependencies among themselves, but test modules should be designed to run independently so that all actions, checks, etc. directly relate to the scope of the test module. This makes it easy to keep abreast of large tests and assure functional changes of a system under test only have a localized effect on the test set.
Each test module contains a list of test objectives – one for each test case. Test objectives serve to further refine the scope of the test module. These allow a reader to understand why test cases are designed the way they are, and give a quick insight in correctness and, more importantly, completeness of a test.
A key benefit of well-designed test modules is readability. Stakeholders with relatively little knowledge of technology or of testing details can easily understand a test. The details needed to execute the test (like which menu item to choose to open a dialog) are not visible in the test module unless they are important to the scope of that test module. Test details are handled by actions, which themselves are maintained in an action library.
The other essential elements are actions. Actions are re-usable containers of a series of operations and checks. Instead of specifying operations and checks in detail in the test module, the tester only has to enter the action and its arguments on a line, Arguments define input values and expected outcomes.
Actions are best regarded as a byproduct of the test design. Testers essentially define the actions and their arguments. While this makes it possible for testers to create both the test and automation, it’s best to collaborate with the automation engineer when developing tests for larger scale automation.
Actions need to be organized and managed well to avoid having more than one action for more or less the same operation. Actions should be clear and unambiguous and should be well documented (meaning, arguments and their default values, etc.).
Action definition in TestArchitect
Actions are characterized in the terms of levels:
- Low-level, UI related, like “click”, “select” (many of these are predefined in TestArchitect).
- High-level, business related, like “enter customer” and “check balance”.
- Intermediate levels, like “enter name and address”, “enter car details”.
Many low-level actions, like “click”, “enter” and “check” come preprogrammed and are located in the action library in TestArchitect for Visual Studio. Low-level actions are application agnostic and apply to a wide range of UI objects. For example the action “click” can activate a button in a window, or it can also activate a hyperlink in a web page.
Intermediate actions typically address one screen in the application under test, and are the building blocks for higher level actions like “enter customer”.
There are a couple of ways to create higher level actions. Actions can be created by programming in C#, but the need to program actions has been minimized. The “action definition” feature allows defining how the new action will work by using existing actions. The action definition feature is the preferred method to create actions that don’t contain complicated technical functionality since it’s much simpler than coding. Replacing automation scripts with keyword-driven actions makes it possible to develop the majority of tests without programming so only the few highly technical test routines need to be coded.
In addition to simplifying the development, execution, and maintenance of tests, TestArchitect for Visual Studio enhances the testing and test-automation development lifecycle. It seamlessly integrates with Microsoft Team Foundation Server (TFS) to define testing based on the same team projects that other areas of the organization use. Test cases created in TestArchitect can be run as part of a test plan in Microsoft Test Manager (MTM) by associating automation with the test cases.
Test automation for today’s software
The sophisticated functionality of today’s software combined with fast-paced agile development environments, compressed release cycles, and platform proliferation makes large-scale software testing extremely complex. TestArchitect for Visual Studio simplifies coded UI testing by replacing coded UI scripts with keyword-driven preprogrammed actions. By speeding the development, execution, and updating of coded UI, programmers and
automation engineers have more time to focus on critical tasks making the team as a whole more productive. Increased test coverage, decreased testing time, and the ability to release software that will deliver the expected customer experience are the benefits that TestArchitect for Visual Studio delivers. ■
About TestArchitect
TestArchitect for Visual Studio is now available for anyone using the Ultimate or Premium versions of Visual Studio 2012. You can request a trial of TestArchitect for Visual Studio by going to www.testarchitect.com.
TestArchitect for Visual Studio is a keyword authoring platform extension designed specifically to enhance coded UI test automation in Visual Studio 2012.