One of the pioneers of the self-driving car movement was Waymo, which grew out of the Google Labs project that developed the sensors and software to provide a completely autonomous driving experience.
Waymo stands for a new way forward in mobility, and its mission is to improve the safety of driving while making it easier for everybody to get to where they need to go. The National Highway Traffic Safety Administrations (NHTSA) classifies the Automation of cars in 6 different levels ranging from 0 to 5; Waymo is currently working on the highest level of autonomy, level 5, in which the vehicle will be capable of performing all driving functions under all conditions. On the other hand, there are also driver assistance controls, such as self-parking systems and adaptive cruise control, which offer some autonomy features but do still require that a driver be ready to take over vehicular controls as needed at a moment’s notice. This type of technology is a function-specifically Automation and limited self-driving-as classified by the NHTSA. As one would expect, testing a self-driving car is far different than testing a normal software application. When testing a normal software application, an expected result can be determined to be right or wrong, but, this is not always the case when testing an autonomous vehicle.
About Waymo’s Technology
In Waymo’s case, testing this technology has been a long process. The technology was first developed in 2009, but the first, fully self-driving test didn’t happen until 6 years later. Waymo cars are equipped with technologies built from the ground up. The cars are equipped with Light Detection and Ranging (LIDAR), a surveying method that sends out millions of laser beams every second, so the car can build a detailed, 360° map of the terrain and objects surrounding it. Radar is also used, so the distance and speed of objects around the car are always known to the car. High-resolution cameras allow the vehicle to identify important things, such as red lights, stop signs, or human beings. Waymo’s cars are able to completely process information around the car as far as 300 yards away. All of this is possible due to the software and machine learning that it was programmed with. Coupled with the software and hardware is the experience of driving over 10 million test miles, which have enabled the system to log various commonalities of pedestrians and other motorists; this has essentially created a library of possible scenarios and allows the vehicle to quickly analyze a situation, recall previously-used possible maneuvers, and anticipate and plan accordingly.
Testing Waymo’s Technology
The main aspects of testing technology like this include making sure the sensors are providing the correct feedback, so the AI can be sure to have the correct information to process. Testing machine learning systems become a little more complicated. Most machine learning systems are based on a neural network, which is nothing more than layered algorithms with variables that are adjusted after training the system. Training the system usually involves experience (i.e. the 10 million miles of driving Waymo has logged). Neural networks have been previously used for image identification, handwriting, speech, and are now finding themselves useful in autonomous vehicle technology. In each case, we need to keep feeding the system data until we are able to arrive at a result that is within our degree of accuracy for known good results. We freeze the formulas at that point and production code can be generated. There are several guidelines that need to be followed for this type of testing:
- We must understand that not every test will produce good results, so there is always going to be a lot of trial and error as we are training the system.
- Know that the results will not always be perfect in this type of testing; there might not be just one correct answer like there usually is in functional testing. Instead, we must live with the fact that the degree of accuracy will not be 100%.
- The architecture of the neural network should also be under scrutiny in the testing process. If input data is consistently giving bad results, the designed models might need a new direction, and we must be willing to adjust it accordingly.
Waymo’s engineers have also created a program called Carcraft-a virtual world where Waymo can simulate driving situations and conditions. Named after the popular online game, World of Warcraft, Carcraft was originally created as a way to “play back” scenes that cars experienced while driving on public roads; for example, a Waymo self-driving vehicle encountered a roundabout on a Texas road, and the complexity of the situation perplexed the vehicle’s software. Now, in Carcraft, the team can simulate numerous, real-world driving situations-such as a car entering the roundabout suddenly. Thanks to Carcraft, 25,000 virtual, autonomous vehicles are “making their way through” modeled versions of cities, including Austin, TX, Mountain View, CA, Phoenix, AZ, and many more! These vehicles collectively “drive” about 8 million miles per day-amounting to about 5 billion total virtual miles as of 2018.
The Future of Waymo
It has been a long and treacherous journey for the self-driving software manufacturer, but each day, they come closer to their vision: fully capable autonomous vehicles. Recently, in an e-mail sent to customers of Waymo One-an autonomous ride-sharing service offered only in Phoenix, AZ-Waymo expressed that in upcoming rides, there may not be a safety driver in the car. Until now, Waymo has always had an employee (dubbed “safety driver”) present behind the wheel of the vehicle in the chance that human intervention is necessary; this announcement of the absence of a safety driver is huge-it means that Waymo is confident enough in its software that they are ready to fully implement autonomy in the car. If successful, we could see Waymo One spreading to other metropolitan areas in the near future but there’s going to need to be a lot of testing and “experience” needed first.
To see how we here at LogiGear are helping to make this vision come to fruition, visit our Automotive Testing Services page. Moreover, visit the LogiGear Magazine website! Our June 2019 issue was focused entirely on Testing the Software Car and features many articles surrounding what is trending in the automotive testing industry.
