AR Technology: ADAS Systems, Automotive Training and Smart Car Development

author: Michael Cheng   

Augmented reality (AR) is predicted to disrupt a wide range of industries, including industrial manufacturing, fashion and auto. The technology can be defined as layers of interactive data or designs that bridge the physical, real world with the fictitious, digital world. Overall, the goals of AR are to blur the lines between both realms and encourage users to communicate with their environment via virtual manipulation or digital tools.

According to a 2016 report by Digi-Capital, a California-based tech research firm that specializes in M&A preparation, revenue from AR/VR products is forecasted to reach $120 billion by 2020. This growth rate is comparable to the timeline automakers have for driverless technology to proliferate and enter commercial markets.

Before AR hits this milestone, it must first overcome several major roadblocks in development. The emerging technology comes with heavy hardware (CPU/GPU) and software requirements, which also come with a hefty price tag. When it comes to hardware, a computer, a display device (head-mounted display [HMD], hand-held display [HHD] and spatial displays [SD]), a tracking system for aligning during movement, applications and interactive tools (controller, touchpad or smartphone) are essential to the functionality of AR platforms. For now, most AR software developers are focusing on the following core aspects of AR (extracted from a case study titled Application of Augmented Reality Techniques in Through-life Engineering Services): optical combination, video mixing and image projection.

Interactive Training and Entertainment

Most early adopters of AR technology in the automotive space are large brands, such as Audi, Ford, Mahindra, Mercedes-Benz and Volvo. This is due to high costs associated with the technology and lack of clear, direct monetization opportunities. A prime example of this is AR advertising. There aren't a lot of AR-focused advertising platforms for businesses in the automotive industry, making car companies leverage AR to boost or add tech-savvy features to their existing products. Many car companies are leveraging AR to boost or add tech-savvy features to their existing products.

Below are some examples of this practice: 

·      Mini (2008, Desktop): Used a webcam and printed paper to display an interactive image

·      Volvo (2012, Tablet): Interaction with the Volvo V40 at the Geneva Auto Show

·      Mahindra (2012, Schermo): Large cheetah interacting with car at XUV500 launch event

·      Volkswagen (2013, Tablet): Component design interface using MARTA System

·      Audi (2014, Samsung Gear VR): Race stimulator presented at the Zurich Film Festival

·      Lexus (2014, Oculus): Cockpit RC F stimulator with interactive features

·      Mini (2015, Custom Glass): Qualcomm-powered AR using custom glass by BMW

Outside of entertainment, AR technology is being used to enhance training in large-scale automotive plants for remote maintenance, diagnostics, non-destructive testing (NDT), as well as repairing and setup activities. In NDT projects, a worker inspects the vehicular component with a compact HMD without touching the potentially defected part. Next, a tablet (HHD) is used to mark the defect virtually on the inspection line. This workflow reduces direct interaction with the target component, which could reduce the possibility of inflicting further damage to the unit.

"AR is a new technology that demands more activity from the user than a normal presentation or text book. In fact, the trained individual needs to look for an image or object in the real world to learn from. This activity demands some action that is supposed to give motivation to complete the activity," said A. Perdikakis, A. Araya and D. Kiritsis, authors of the paper Introducing Augmented Reality in Next Generation Industrial Learning Tools: A Case Study on Electric and Hybrid Vehicles.

For technical training purposes, AR-based curriculums offer more "hands-on" opportunities, compared to traditional classroom-types of learning. An instructor could upload a virtual car, for instance, for workers to interact with before its commercial release. It would also be possible to re-create certain scenarios where there is engine or vehicular component failure for workers to troubleshoot and master. Such robust platforms can improve productivity by offering a customized approach to on-the-job training. With AR headsets readily available, workers could revisit previous lessons anytime/ or anywhere without needing physical access to the company's training facility. Although AR technology is considered to be expensive, when applied in automotive training, adoption and deployment costs are significantly lower, compared to classroom-type curriculums. This application is feasible at almost all parts of automotive career development, from entry-level mechanics to senior engineers interested in supplementary learning.

Driver Safety, HUDs and Autopilot

AR technology is currently being used to supplement safety features in vehicles. Continental recently introduced a prototype AR-HUD system for assisting drivers, which will go into production in 2017. The platform works by displaying actionable data and images on the vehicle's windshield. There are two AR layers available that aids in the following maneuvers: turn-by-turn route navigation, lane-departure warning (LDW) and adaptive cruise control (ACC). The three driving executions are considered to be the most relevant maneuvers today for cars on public roads, according to Continental researchers. The AR-HUD unit is connected to the car's Advanced Driving and Assistance System (ADAS), which includes sensors, cameras, GPS and etc.

"These ‘layers' (imagine two vertical planes, one closer than the other) are created by two different image generation units. One unit consists of a thin-film transistor, a smaller optical mirror, and a LED array; it produces an image ‘near' to the viewer (in the prototype this was the base data such as road speed). Its field of vision of 5°x 1° (corresponding to 210 x 42 mm/8.2 x 1.65 in) has a projection distance of 2.4 m (7.8 ft)—roughly the equivalent of a conventional HUD," said Dr. Thorsten Kern, Manager of the HUD Competence Center at Conti's Babenhausen interior-electronics development complex.

In application, the driver may see blinking arrows when the system senses an upcoming turn during navigation. This interactive image only appears within the peripheral vision of the user in order to reduce the possibility of distraction. In the event the car drifts out of its designated lane, the system will detect the movement and initiate an LDW by causing the seat cushion to vibrate and display a series of red dots, marking the violated lane on the augmented windshield. During ACC activation, a virtual arc is displayed on the windshield, indicating the distance between the AR-powered vehicle and nearby cars. Closing the distance too quickly will initiate a red warning, while a blue arc represents a safe following distance.

Barriers-to-Entry and Consumer Adoption

AR technology still has a long way to go before it reaches acceptable levels of maturity and commercial use. Before this can happen, developers must overcome several technical and developmental obstacles. A 2016 survey by Perkins Coie and Upload, which included 653 respondents (AR/VR executives, founders and investors), revealed the core issues holding back the industry:

·      Low quality content: 37 percent

·      Regulatory/legal risks: 4 percent

·      Technology and intellectual property licensing: 19 percent

·      Product liability and safety: 18 percent

·      Consumer privacy and data security: 15 percent

It is crucial to clarify that the issues above are related to business or backend development of AR products. For consumers, most of the concerns above are not relevant (except for content and privacy-related matters). When it comes to adoption of AR technology, user experience was cited as the biggest challenge for developers, at 35 percent for AR and 38 percent for VR. Cost of AR products was the second major concern that could hinder adoption at 32 percent for VR and 25 percent for AR.  

From a long-term perspective, AR investors have expressed the need for a more established market (80 percent of survey respondents view this as the greatest concern). To initiate growth, businesses in the industry will engage in more capital funding rounds (42 percent), look for acquisitions (25 percent) and form partnerships (22 percent). When it comes to the amount of investments, a reassuring 88 percent have already contributed over $5 million to AR expansion in 2016.

The Road to True AR

Many scientists and engineers argue that today's AR platforms are a mix of AR and VR. The establishment of true AR, which will come after the maturity of both AR and VR, requires advancements in manipulating controlled matter and reconfiguring atoms in order to change the user's environment. In a 2015 study titled Breaking the Barriers to True Augmented Reality, researchers highlighted this concern as "displays that use magnetic fields to rapidly create shapes out of ferromagnetic fluid." The overall objective of true AR is to merge reality with virtual elements. Based on this goal, it is clear that current AR technology is lacking foundation in light field displays, data processing and realistic visual components. Today's AR devices, like Microsoft's HoloLens and Magic Leap's AR prototypes, still make very obvious distinctions between the virtual world and real-world environments.

To conclude, in the automotive space, signs of AR technology maturity includes deeper applications in ADAS systems and passenger safety features. At the moment, due to the limitations of AR, automakers are only using the technology for entertainment and in controlled environments, such as internal training programs. In order to hone the technology for smart cars, some businesses in the sector have expressed their willingness to work with AR-focused companies, like Gorilla Glass maker Corning. Lastly, another sign that AR technology has finally reached mainstream status is integration with city infrastructure (vehicle-to-infrastructure and vehicle-to-vehicle protocols) and consumer gadgets.

Michael Cheng

Michael Cheng is a legal editor and technical writer with publications for Blackberry ISHN Magazine Houzz and Payment Week. He specializes in technology business and digesting hard data. Outside of work Michael likes to train for marathons spend time with his daughter and explore new places.