What is often misunderstood is that automated testing is not the same as automating manual testing. For automated testing to be successful, tests need to be designed with their automation already in place. This source of many test automation failures is the absence of this simple factor.
To tackle automated testing we have come up with a vision that consists of the following elements:
- An effective keyword-driven method for test design and automation called Action Based Testing (ABT)
- A toolset to support the method that has all the features commonly found in test tools including a shared repository organization and flexible web-based “dashboards” for managing test projects.
The ABT method was a concept devised 16 years ago as a complex yet very critical project that is now widely used in software testing. The tests are clustered in “test modules”, and are described as a series of “actions”, varying from inputting a value, clicking a button or capturing and verifying a value. The actions are named with “action keywords” or simply “action words” and have arguments for the input and expected output values. Test modules also include a set of “test objectives” that detail the goals of the test.
A key concept of ABT is that the automation effort is focused solely on the actions as named by their action words. This results in tests that are readable for humans and at the same time are fully automated. In most projects a relatively small set of actions can be used to describe a large set of tests. This makes the automation very maintainable. In the case of a change in the system under test, typically only the implementation of one or more actions has to be modified and the majority of tests can be left alone.
TestArchitect TM is the name we have given to our automation toolset. It reflects the vision that automated testing requires a well-designed architectural plan allowing technical and non-technical elements to work fluidly in their capacity. It also addresses the continual missing link of all test automation tools of how to design tests. In TestArchitect the test design central to developing efficient large-scale test automation.
TestArchitect was developed to facilitate ABT test development. It organizes the test modules and the actions, and has tools similar to a test editor to quickly create the action based tests. It also features an “action definition” function to allow existing actions to be used to create new ones without additional programming. The subscriptions feature allows actions to be shared across test projects by having a project “subscribe” to another “supplier’” project. The subscriber can then use all actions of the supplier project freely and any project the supplier itself subscribes to are automatically available to the subscribing project. Subscriptions can be recursive and mutual: a supplier can subscribe to yet another project, in which case those actions are available for its subscribers as well, and even a supplier project can subscribe to one or more of its own subscribers.
The underlying concept in ABT is to design tests explicitly and to not rely on recording of tests. This is beneficial in Agile environments when its advantageous to begin test design in parallel with project development. TestArchitect also come with a recording tool called the Action Recorder. Instead of generating scripts this tool produces action lines that can be inserted anywhere in a test module or action definition. The Action Recorder allows extremely fast test creation when the UI for the SUT is already developed.
Another important aspect of TestArchitect is test result management. After a test runs, results are initially stored locally. The user must take an explicit action to store them in the repository visible to others and later have an impact on metrics and reports. This process prevents repositories cluttered with results of the numerous ad hoc dry-runs users supervise during testing or automation development. Once in the repository, results are further organized with proper naming or storage placement. Lastly, results are linked to the test modules and test cases they have tested.
Automation in TestArchitect is organized as a set of libraries that work completely separate from the test management and test design components making it possible to program the actions in virtually any programming language. It is also possible to execute the tests in a third party playback tool using its scripting language to program the actions.
All in all, TestArchitect is an ecosystem for testers to design, automate, and manage tests grounded in its core design of Action Based Testing. Without the ABT method it would be just another playback tool similar to others on the market. It is our belief that TestArchitect makes it possible to improve the quality and implementation of tests. This is at the very core of our primary objectives as testers: to write great tests that find meaningful bugs and help improve overall application quality.
