QA Localization is more than just translation.
This is part two of a two-part series on Testing Strategy for QA Localization. Part 1 discussed the relationship between Globalization, Internationalization, and Localization as well as a few types of Localization testing. Part 2 will discuss how to develop your testing strategy for QA Localization.
Developing Your Testing Strategy for QA Localization
Is Classifying Tests Really That Important?
Yes, it actually is. For communication, transparency, coverage, analyzing risk, test planning–all the things that makeup strategy–making sure that your team communicates correctly and shares a mutual understanding is absolutely essential. Test types are also the easiest way to define a test strategy.
But just as importantly, these different testing tasks are often done by different people at different times during development. This will define your testing strategy and needs planning. It also often relies on outside resources that need to be planned and scheduled independently in the test strategy.
It is important to understand the various Software Testing types and Software Testing tasks of Globalization, Internationalization (I18N), and Localization(L10N). This has everything to do with test strategy and test planning.
Strategy and Planning
We know finding bugs early is cheaper, faster, and easier. You want to find as many functional issues as you possibly can in the core product by your own technical test team. The localized product testing staff may, in many cases, be a more expensive and outsourced translation staff. When you find bugs in the individual localized product, everyone has to stop what they’re doing. Run regression tests against other languages to see whether the issue found is unique to that language or goes across the whole product. Then you have to take a step backward, fix the main product, then do a build across every language the product is going on and start again.
One time I was working on a product-software and hardware-where we took the core product that had been already released in the US market, and it was separately internationalized and we made an International English version and did a bunch of functionality testing. After we thought we were “done,” it was translated and localized for FIGS—French, Italian, German, Spanish and Portuguese—and a DB project for Japanese, both Chinese and Korean.
After the hardware was built, unique to each locale/market, a bug was found in the German version. And it was a reproducible bug. Ugh.
Everyone stopped and immediately tried to reproduce that bug on the other builds. There was a lot of spins and immediate change of focus. In this particular instance, the bug did not show in the other four builds… right away. We thought it was a German specific bug and brought it to the attention of the developers who said: “if that’s how it shows in German, it has to be in the other builds. It took a bunch more testing and a bunch more spin, but we wound up finding that same bug on a couple of the other builds. It showed up differently in the other products. It showed up first in German. We went back to the International English version and after messing around for a while we found it there also. Like bugs sometimes do, it caused a lot of distraction and re-testing and re-testing and re-testing across versions/builds which is the nature of Software Testing – but more than once someone brought up “Why didn’t we find this in the one international English version before the 10 separate builds?” It would have caused fewer problems and taken less time. Do you ever find *every bug*? That is a debate for another article. Does testing miss some bugs? Yes, even the best teams might miss a bug that makes it into the individual language builds. But, you need to minimize this as much as possible because of the cost of fixing.
This case does highlight the point—it would absolutely have been easier, faster, and cheaper to spend some extra time testing the International English version, and fixing issues in it before 10 sets of ROMS were burned and hardware systems built for the final localized Software Testing. The testing, in this case, on the International English version was done by one team from the home/main development office. The 10 sets of localized hardware/software systems were tested by a much larger, different language/locale skilled team.
So clearly, you need to find as many low levels and functional bugs in the core product before it gets localized. They are more expensive to find later in individual languages.
Defining your test strategy, for example, as:
- 1st- functional, ability to localize testing done by the home team
- 2nd- locale, language, UI, cultural appropriateness
…maybe an easier, more cost-efficient strategy than first focusing on translation or UI issues.
The goal is to have all of the Internationalization functionality worked out in the core product. The Localization type errors—Menu size, word choice, cultural appropriateness, truncated strings—can be tested on an individual language/locale basis. By the point of translation, it is too late to find, for example, date format issues. These should be found much earlier.
The types of issues common for Localization testers to find should be more specific to individual languages. Issues that you find during the Localization Testing phase should be similar to UI bugs. Cleaning up issues.
| Understand the Complexities of Date Formats |
|---|
| Here is an example of long and short dates within different regions of the world: Available date formats for English (United States) M/d/yy 4/24/20 M-d-yy 4/24/20 MM/dd/yy 04/24/20 MM-dd-yyyy 04/24/2020 dd-MMM-yy 24-Apr-20 yy-MM-dd 20/04/24 yyyy-MM-dd 2020/04/24 For US-English, there are 7 available, *standard* date formats only for short format. There is not a simple one date format, with many more for the long format. An example from Microsoft of date formats from just 3 countries: Long Date (Tuesday, October 12, 1954) Short Date (10/12/54) Now let’s compare these formats between the United States (English), Mexico (Spanish), and Japan (Japanese). Long Date United States: Tuesday, October 12, 1954 Mexico: martes 12 de octubre de 1954 Japan: 1954年10月12日 Obviously, the names of the months and days of the week are different from locale to locale. However, there are other notable differences as well. For Mexico, the day comes before the month, everything is lowercase and the article “de” has been added. In Japan, the day of the week is not displayed, and the translations for day, month, and year act more like separators. Short Date United States: 10/12/54 Mexico: 12/10/54 Japan: 54/10/12 In the short date, we again see that in the Mexican example the order is day/month/year as compared to the United States where it is month/day/year. In Japan, the order is year/month/day. This can cause confusion if not watched carefully. For example, depending on which location you are in, the date specified as 07/04/01 could mean: United States: July 4th, 2001 Mexico: April 7th, 2001 Japan: April 1st, 2007 These examples show why you should use the date APIs when dealing with dates. Not only will they handle the correct format, but they will also display the correct translations for the long dates, thus saving translation costs. |
Locale Settings
For more information on date formatting, review this documentation from Microsoft.
As I mentioned earlier in the introduction, I worked on a testing project for a food safety system. None of the above date formats were even considered. We followed only the date format standard for how expiration or shelf-life was printed on food labels for March 7, 2020 expiration date: 2020 MA 07 or 20 MA 07. There was also a *standard* to be written as 20200307. You can see Localization is about a lot more than translation.
Before you get overwhelmed about international standards or how extensive I18N and L10N testing can be, most Test Engineers will follow what the Product Owner specifies and the Software Engineers codes for. You will know if sort order, paper size, and address format are important to your product or not.
The whole team must closely collaborate with what locale settings are important for testing.
For the skills you need, of course, know as much as you can about locale settings-you will do better testing, but OSs, devices, browsers, supported platforms, languages, markets, settings will dictate the scope of testing for your particular product.
Multilingual Automation for Localization
Localization can also include multilingual automation as part of its core strategy.
It is unique in a Localization project to focus on Automation from the start—but this is the best strategy. Having a well-structured multilingual Automation architecture will make the test cases more extensible and easier to maintain. Building test modules for I18N and L10N in mind makes your life easier. To learn more about automating multi-language testing, refer to the article “These are 6 Considerations for Automating Localization Testing” written by Hans Buwalada.
In the past, I have worked on many development projects that did Localization Testing manually:
- They were one-off projects and the value of automating was not there.
- The Automation tool could not deal with different character sets.
- It was too hard!
- This was just more tests to maintain
But the world has changed. With iterative development styles, global products are in constant development and need Continuous Testing. Some Automation can now easily handle Double-byte characters the same as they would handle English characters. If you use good Automation test design, creating an extensible framework where language/characters are just another variable or validation point and not an entire set of their own scripts to write and maintain, it’s not as complicated if you know how to do it. Plus, scaling to automate many languages can have an exponential effect reducing the manual test effort. It’s a great cost-savings to automate much of your Localization Testing.
Summary
Understanding what Localization Testing is about and the correct terms used along with it will have an immediate impact on the clarity of communication and understanding of risk. Also, building a cost-effective strategy is reliant on knowing the focus of different test types through the engineering and translation phases. You want to test the internationalized functionality of the product/application first and uncover as many functional bugs before you localize. Localization is more than just date and currency formatting. They are actually Internationalization/functional tests. Don’t get hung up on the translation-it is important, but often the last, and easiest step. For many reasons, Test Automation did not used to be a focus for Localization projects-today it is a lot easier and definitely cost-effective.
Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), available in English, Chinese and Japanese, and Global Software Test Automation (HappyAbout Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries."