Reprinted from Catalyst, December 6, 2024

Since the 2004 DARPA Grand Challenge, connected and autonomous vehicles (CAVs) have been highly anticipated and widely discussed. Today, Teslas with “autopilot” and General Motors vehicles with Super Cruise driver-assistance technology are already on roads, and pilot “robotaxi” services operate in several major US cities.

However, most CAVs are currently classified, at best, as Level 4 by the Society of Automotive Engineers. This means they are designed and operated with specific, predefined conditions—known as their operational design domain (ODD)—and must stop safely when those conditions are no longer met. Despite advancements in artificial intelligence and machine learning, there is still a long way to go before fully autonomous, or Level 5, vehicles become a reality.

Partial remote driving, or teleoperated driving (ToD), has emerged as a potential interim solution. With ToD, a remote operator can take control if a CAV encounters conditions beyond its ODD. Enabled by 5G cellular networks, ToD has shown promise in controlled settings, but the question remains whether current 5G networks can reliably support remote driving on a large scale.

In a recent project, University of Minnesota researchers investigated the feasibility of and critical networking requirements for remote CAV operation. The project was led by Zhi-Li Zhang, a professor in the Department of Computer Science and Engineering, and Rajesh Rajamani, a professor in the Department of Mechanical Engineering. Their work was supported by CTS seed funding, which aims to help CTS scholars develop expertise in emerging areas and foster strategic relationships that position them for future funding opportunities.

According to Zhang, 5G was designed to enable low-latency applications—those that process high volumes of data with minimal delay. In reality, today’s commercial 5G networks mainly support conventional mobile broadband access, especially to improve download speeds. But when it comes to teleoperation, higher uplink speeds and low latency in both directions are essential, Zhang says.

To test 5G’s potential, the research team used the MnCAV Ecosystem’s research vehicle—which is outfitted with cameras and lidar sensors—to conduct repeated driving experiments on commercial 5G networks in downtown Minneapolis. The study focused on end-to-end uplink performance of sensor data from the vehicle to a remote teleoperation station, analyzing how well these networks could support responsive, safe control.

Results showed that while transmitting a single video stream from a CAV is feasible, adding additional streams, especially from lidar—essential for depth perception—can strain the network. The researchers also found that, even in the case of a single video stream, latency increased when the vehicle was traveling at higher speeds and at handover points between 5G base stations, posing risks for safe and reliable remote driving.

These findings highlight fundamental challenges for remote driving on commercial 5G. However, thanks in part to this CTS-funded project, Zhang, Rajamani, and other researchers from the University of Minnesota and the University of Michigan were awarded an NSF grant to study further solutions.

One approach the researchers are exploring in this project is a new “predictive display” mechanism that leverages generative artificial intelligence to overcome the latency challenge of 5G networks. The mechanism uses recent but slightly delayed (e.g., by 0.5 seconds) data to predict the CAV’s current surroundings. Early tests suggest that this method could improve remote driving performance by masking the 5G network delay, helping teleoperators drive more effectively. However, the researchers say further work is needed to refine the technology and make remote CAV operation reliable and robust at scale.

—Krysta Rzeszutek, CTS digital editor

Related research by MnDOT

• Influence of Autonomous and Partially Autonomous Vehicle on Minnesota Roads (2023)
• Designing an Autonomous Service to Cover Transit’s Last Mile in Low-Density Areas (2024)
• The Grand Iron Range CAV Initiative: History, Partnerships, and Community Engagement (2024)