Here's How Waymo Uses Hyper-Local Weather Data to Improve the Capabilities of its Self-Driving Vehicles

author: Eric Walz   

Rain, fog and condensation can affect how well an autonomous vehicle's sensor suite performs.

Traveling in inclement weather can be tough on some drivers. Driving in the rain for example can reduce visibility, fog a vehicle's windows or generate unwanted glare from a wet road surface. 

But for autonomous vehicles and robotaxis that will one day operate without a human behind the wheel, dealing with bad weather is a big challenge. It is one of the reasons why developers of self-driving vehicles are using sophisticated weather data to help them to operate more safely.

Alphabet's autonomous driving unit Waymo, for example, is getting ready to scale its autonomous ride-hailing service in both San Francisco, California and Phoenix, Arizona before rolling out to new cities in the U.S. In a recent blog post, Waymo shared some of the ways its autonomous robotaxi vehicles deal with rain, fog and other poor weather conditions. 

In San Francisco, where Waymo recently launched its robotaxis service, the fog that rolls in from the Pacific Ocean each evening is something that the company's autonomous vehicles must be able to handle.

A self-driving vehicle relies on perception sensors, similar to how a human driver uses their eyes and ears to safely navigate. However, poor weather can be challenging for both humans and can impact autonomous driving systems.

For example, condensation can alter sensor data and fog on a camera lens can degrade its performance. In addition, water droplets, road dirt and ice can collect on the surfaces of vehicle sensors and degrade their perception capabilities, while wet roads can become more slippery. For Waymo's autonomous vehicles that rely on software, overcoming these challenges is vital for the safe operation of its commercial robotaxis that will carry passengers.

Waymo's latest hardware, including cameras, radar, and lidar, use raindrops on a vehicle's windows to classify various weather conditions. This data is then combined with high-quality ground-truth data from weather visibility sensors. All of this information is used by Waymo to generate a quantitative metric about meteorological visibility.

Waymo says this metric is used to conduct an analysis of the weather around one of its self-driving vehicles in real-time, which essentially makes each Waymo robotaxi vehicle an autonomous "mobile weather station".

Although ground-based weather stations and satellites provide meteorologists with accurate data to make fairly accurate weather predictions,  Waymo says that these high-tech weather tools and datasets often lack the precision and specificity to accurately reflect the conditions where its vehicles are driving.  

For example, some of this doppler radar weather data is measured at elevations about 3,500 feet and as high as 9,000 feet. This may not match the current real-time weather conditions on the ground where Waymo's self-driving vehicles operate. In addition, many weather stations are located near major airports and are not always the ideal best source for real-time, local weather information. 

San Francisco's weather can vary greatly from neighborhood to neighborhood as locals can attest to. Residents on the east side may experience sunshine while the city's Outer Sunset district may be experiencing dense fog at the same hour of the day. 

While remote weather data from satellites or radar may help fill in the gaps from local weather stations, they also do not directly sense weather conditions on the ground where Waymo's vehicles are operating, especially when its cloudy, according to Waymo. So the company's fleet of autonomously driven vehicles serve as mobile weather stations combining millions of data points to more accurately predict weather on the ground for the entire fleet.

Doppler radar does not always reflect the actual weather conditions on the ground, where autonomous vehicles operate.

This additional layer of weather data is being used for Waymo's ride-hailing services in San Francisco and Phoenix, and the company said its will create similar weather maps for additional cities as it scales its Waymo One autonomous ride-hailing service.

Waymo even created the first-of-its-kind "fog map" for San Francisco that provides an unprecedented amount of spatial-temporal resolution. It allows the company's autonomous fleets to track the progression of coastal fogs as it flows in daily from the Pacific Ocean and burns off as the sun rises later in the morning. 

Waymo's weather sensing autonomous vehicles can even detect drizzle and light rains that are invisible to the National Weather Service's local Doppler weather radar. This detailed weather data allows Waymo to localize where the weather conditions are beginning to deteriorate or where it is improving. 

This detailed understanding of the current weather and its impact on Waymo's autonomous driving technology stack present additional opportunities for designing fleets of robotaxis that will frequently encounter varied weather conditions. For one, Waymo's engineers can refine the Waymo Driver's performance in a variety of weather conditions using various data augmentation and simulated environments representing the seasons of the year.

Since Waymo uses the same sensor suite to autonomously navigate and collect weather data, the company will build an extensive dataset that can be used to further improve its weather simulation capabilities. 

Waymo's understanding of the weather is also being used by its autonomous freight delivery business Waymo Via. In the future, it will be used to help Waymo Via's delivery partners and their customers by providing more accurate updates regarding their cargo while in transit, especially for long-haul freight deliveries across greater distances that span multiple states and geographic locations.

Eric Walz

Originally hailing from New Jersey, Eric is a automotive & technology reporter covering the high-tech industry here in Silicon Valley. He has over 15 years of automotive experience and a bachelors degree in computer science. These skills, combined with technical writing and news reporting, allows him to fully understand and identify new and innovative technologies in the auto industry and beyond. He has worked at Uber on self-driving cars and as a technical writer, helping people to understand and work with technology.