A Test Scenario, A Bug Report Deferred As-Designed: 437,000 Toyota Priuses Recalled

According to Associate Press (February 9, 2010, Yuri Kageyama, AP Business Writer), “Toyota is recalling 437,000 Prius and other hybrid vehicles worldwide to fix brake problem.” Toyota is the world’s largest automaker with impeccable quality reputation up to now.

As it turns out, the cause of the problem is a bug in the software that controls the brake system. “There have been about 200 complaints in Japan and the U.S. about a delay when the brakes in the Prius were pressed in cold conditions and on some bumpy roads. The delay doesn’t indicate a brake failure. The company says the problem can be fixed in 40 minutes with new software that oversees the controls of the antilock brakes.”

I don’t mean to wage in on Toyota disastrous situation. I just think that this is an essential software testing lesson and it illustrates some key issues that testers like us have experienced over the years. “After receiving a similar report from the U.S. in October, Toyota’s tests concluded that a glitch in the Prius antilock brake system software could reduce braking force when drivers traveled across bumpy surfaces. The company wrote that “although this system was operating as intended,” it decided to make a change to its production line in January to address the problem,” Kareyama reported. If I wrote a bug report, it would look like this:

• Bug Report Summary: Braking force is reduced (the problem) when driving across bumpy surface (the scenario).
• Reproducibility: Intermittent

It seems that the Resolution can be treated as “As-designed” (although it does not work in certain scenarios, that was how Toyota designed it), or “Deferred” (the problem is acknowledged and will be fixed later), or “To-be-fixed” (but in the next release). We already know the outcome. So, what do we learn?

• Scenario-based and exploratory testing during system-test is essential—this requires skill and creativity in test design, not test-driven development (which is all good and an essential element to software development) or process-driven testing such as CMMi or ISO-9000 (which is good for quality control).
• Software testing is not quality assurance—testers can report bugs, but a decision for corrective action lies somewhere else. There is nothing wrong with that. Testing is an important information service provider. We find and report problems.
• Software is everywhere; and software is buggy—our job as testers is to find bug by breaking software in every possible way we can!

Last but not least, we, testers don’t create software or product; we help improve software reliability through our bug finding skills. Buy breaking software, we are saving consumer and our company time and money (even contributing to public safety in this case). So, keep breaking software tester…Your job is much appreciated!

Share:

Hung Nguyen

Hung Nguyen co-founded LogiGear in 1994, and is responsible for the company's strategic direction and executive business management. His passion and relentless focus on execution and results has been the driver for the company's innovative approach to software testing, test automation, testing tool solutions and testing education programs.

Hung is co-author of the top-selling book in the software testing field, "Testing Computer Software," (Wiley, 2nd ed. 1993) and other publications including, "Testing Applications on the Web," (Wiley, 1st ed. 2001, 2nd ed. 2003), and "Global Software Test Automation," (HappyAbout Publishing, 2006). His experience prior to LogiGear includes leadership roles in software development, quality, product and business management at Spinnaker, PowerUp, Electronic Arts and Palm Computing.

Hung holds a Bachelor of Science in Quality Assurance from Cogswell Polytechnical College, and completed a Stanford Graduate School of Business Executive Program.