Advanced Driver Assistance Systems (ADAS)

What is Advanced Driver Assistance Systems (ADAS)?[1]

Advanced driver assistance systems (ADAS) are systems developed to automate/adapt/enhance vehicle systems for safety and better driving.

ADAS employ ultrasonic, radar, laser, camera, thermal and infrared sensors to monitor the world outside the vehicle. The information gathered is used by onboard computers to take appropriate actions when potentially hazardous situations arise. That could be something as simple as illuminating a warning light when a car enters the driver’s blind spot, or something far more complex such as taking control of the car’s throttle, brakes and/or steering to avoid a collision.[2]

Advanced Driver Assistance Systems (ADAS) Developments[3]

On March 31, 2014, the US Department of Transportation’s National Highway Traffic Safety Administration (NHTSA) announced that it will require all new vehicles under 10,000 pounds (4,500 kg) to have rearview cameras by May 2018. The rule was required by Congress as part of the Cameron Gulbransen Kids Transportation Safety Act of 2007. The Act is named after two-year-old Cameron Gulbransen, who was killed when his father failed to see the toddler and accidentally backed his SUV over him in the family’s driveway. GM offers vibrating seat warnings, in Cadillacs starting with the 2013 Cadillac ATS. If the driver begins drifting out of the traveling lane of a highway, the seat vibrates on the side of the seat in the direction of the drift, warning the driver of the danger. The Safety Alert Seat also provides a vibrating pulse on both sides of the seat when a frontal threat is detected. The system was first offered by Citroen in 2006 as part of its AFIL (Lane Departure Warning) system. Alcohol ignition interlock devices do not allow the driver to start the car if the breath alcohol level is above a prescribed amount. The Automotive Coalition for Traffic Safety and the National Highway Traffic Safety Administration has called for a Driver Alcohol Detection System for Safety (DADSS) program to put alcohol detection devices in all cars.

Passive and Active Advanced Driver Assistance Systems (ADAS)[4]

Advanced Driver Assistance Systems (ADAS) range on the spectrum of passive/active.

  • A passive system alerts the driver of a potentially dangerous situation so that the driver can take action to correct it. For example, Lane Departure Warning (LDW) alerts the driver of unintended/unindicated lane departure; Forward Collision Warning (FCW) indicates that under the current dynamics relative to the vehicle ahead, a collision is imminent. The driver then needs to brake in order to avoid the collision.
  • In contrast, active safety systems take action. Automatic Emergency Braking (AEB) identifies the imminent collision and brakes without any driver intervention. Other examples of active functions are Adaptive Cruise Control (ACC), Lane Keeping Assist (LKA), Lane Centering (LC), and Traffic Jam Assist (TJA). ACC automatically adjusts the host vehicle speed from its pre-set value (as in standard cruise control) in case of a slower vehicle in its path. LKA and LC automatically steer the vehicle to stay within the lane boundaries. TJA is a combination of both ACC and LC under traffic jam conditions. It is these automated features that comprise the building blocks of semi/fully autonomous driving.

Advanced Driver Assistance Systems (ADAS) Features[5]

Advanced Driver Assistance Systems (ADAS) Features were once offered only on high-end vehicles, but today they are available on a growing number of mid- and even entry-level models.

  • Lane Departure Warning (LDW): An LDW system warns a driver on the highway if their own car may depart from the current travel lane without a lane change signal by detecting traffic lane markings from image data of a front or rear camera.
  • Traffic Sign Recognition (TSR): A TSR system recognizes traffic signs using image data from a front-mounted monocular camera and displays the recognized information of the recognized traffic signs on a display panel. This system may be used to compare the recognized speed limit with the vehicle speed read via CAN and then it notifies a driver.
  • Forward Collision Warning (FCW): An FCW system detects vehicles in front of the own car by using image data from a stereo camera or a monocular camera with milliwave radar at the front of the own car and warns a driver of a potential collision risk, based on the distance between both vehicles and the own vehicle's speed. A BCW (Backward Collision Warning) system detects vehicles in the rear of their own car by using image data from a rear monocular camera.
  • Lane Change Assistance (LCA): An LCA system detects obstacles such as nearby vehicles for lane change assistance and warns the driver of the danger of hitting them. It uses a monocular camera at the rear or two monocular cameras (at the rear-right and rear-left).
  • Obstacle Detection: An obstacle detection system detects obstacles such as vehicles around own car for lane change assistance and forward collision warning. It uses three monocular cameras (one at the front and one each at the rear-right and rear-left).
  • Right/Left Turn Awareness: A left/right-turn awareness system detects pedestrians, bicycles, and motorcycles using cameras mounted on either the left or right side of the own car and warns the driver of the danger of hitting them.
  • Forward Pedestrian Collision Warning: A forward pedestrian collision warning system detects pedestrians with a front-mounted camera and alerts a driver. An infrared camera is used in the nighttime (Such a system is also called Night Vision).
  • Backover prevention (backward pedestrian collision warning) system can be realized by mounting a camera at the rear of own vehicle.
  • Driver State Monitoring (DSM): A DSM system tracks the driver's facial direction and notifies if the driver is not paying attention ahead.
  • Parking Assistance with Bird's Eye View: This system captures images around own vehicle by using four fish-eye or wide-angle lens cameras and converts these 4 images to bird's-eye viewpoint images and displays synthesized image from the above 4 converted images and a top view image of the own vehicle to assist when parking.
  • High-Beam Assistance: This application detects the headlights of oncoming vehicles and the taillights of leading vehicles to automatically select an appropriate lighting range from the high and low beams.
  • Traffic Signal Recognition: This application detects red traffic lights and warns the driver if a vehicle shows a sign of going through a red light.
  • General Stationary Obstacle Collision Warning: This application detects general obstacles on the road, including fallen objects, rockslides, landslides, and traffic cones, and warns the driver when there is a danger of collision.

Advanced driver assistance systems
source: QNX

Some ADAS features are already well-known and provide welcome convenience and safety. These include adaptive cruise control, blind-spot monitoring, lane-departure warning, and night vision. The more advanced, and sometimes controversial, ADAS features are the ones that actively help drivers avoid accidents. Unlike seatbelts and airbags that mitigate the effects of a crash, these ADAS features act preemptively. Instead of only decreasing injury or improving your chances of survival in an accident, some ADAS features are designed to prevent an accident from happening in the first place, in some instances by taking control of the car away from the driver. These include collision avoidance systems that can automatically apply a car's brakes and lane-departure prevention to steer a vehicle back on track.[6]

Advanced Driver Assistance Systems (ADAS): Awareness & Safety[7]

Advanced Driver Assistance Systems (ADAS) continue evolving to deliver not only improved passenger experience and comfort but provide optimum safety to the driver. Enabling better situational awareness and control to make driving easier and safer, ADAS technology using FPGA/SoCs and automotive sensors can be based upon systems local to the car, i.e. “vehicle resident systems” like vision/camera systems, sensor technology, or on smart, interconnected networks as in the case of vehicle-to-vehicle (V2V), or vehicle-to-infrastructure (V2I) systems, jointly known as V2X systems. V2X communications use onboard dedicated short-range radio communication devices to transmit safety-related messages about a vehicle's speed, heading, brake status, vehicle size, etc. to other vehicles and receive the same information about other vehicles from these messages. Using multi-hops to transmit messages through other nodes, the V2X network can communicate over long distances. This longer detection distance and ability to “see” around corners or through other vehicles help V2X-equipped vehicles perceive some threats sooner than sensors, cameras or radar can, and warn their drivers accordingly. Solutions in conjunction with V2I, the potential safety advantages of a wide-scale implementation are enormous. Potential V2I safety applications include:

  • Red Light Violation Warning
  • Curve Speed Warning
  • Stop Sign Gap Assist
  • Reduced Speed Zone Warning
  • Spot Weather Information Warning
  • Stop Sign Violation Warning
  • Railroad Crossing Violation Warning
  • Oversize Vehicle Warning

ADAS Security Problem Areas[8]

In general, any malicious actions that could cause the ADAS system to behave outside its specification are referred to as threats to ADAS. And the interfaces that allow such threats to occur are referred to as attack surfaces. Now the key questions are: what is the specified behavior of an ADAS system, and how do attackers cause the system to misbehave? The answers to these questions lead to the discovery of three major ADAS security problem areas.

  • Control System Security: ADAS can be thought of as a closed-loop control system. While in operation it must satisfy functional safety, efficiency, performance, and reliability requirements. This whole system behavior is referred to as “ADAS control system and processing” and makes securing it our priority. Threats to the control function come from any actions the attacker could take, given their capabilities, to cause the system to act outside of its specifications. Any changes in the system properties that contribute to the violation of its safety goals may happen due to deliberate attacks. The security requirements to support safety goals should mainly concern establishing and maintaining functional integrity and other requirements. With further threat analysis and risk assessment, one could derive detailed security requirements.
  • ADAS Data Protection: In addition to attacking core functions of ADAS, the attacker could be motivated to attack the system to achieve other unexpected consequences by the original design. For example, the attacker could eavesdrop on ADAS data processing and/or internal communication to gain access to ADAS data. Leakage of data to an external party other than for local control system consumption may also be an unexpected behavior of the system, therefore a second area of security problem for ADAS may come from ADAS data protection. A security system must ensure Confidentiality, Integrity, and Availability (CIA) of data collection. The practice of recording data for accountability may be implemented by a “BlackBox” system. Should the BlackBox be implemented, storage security would be an issue. Similarly, integrity protected storage system would be required if the storage is used for collecting and storing other vehicle-related information, such as object classifiers or maps. The specific threats are similar to those seen in a typical storage system in the traditional computing world.
  • Secure Lifecycle Management: Deploying and maintaining intended modules in the ADAS is as important as any other protection mechanism for ensuring ADAS system behavior according to the specifications. This process is typically referred to as lifecycle management.

Changes to the ADAS system could be triggered by: • System upgrade/algorithm updates • Software patch • Installation of new components for additional functions • Hardware recovery and replacement • System recovery due to compromise • Root of trust update due to authority updates in the administrative domain • Cryptographic algorithm and key updates due to cryptosystem migration or other reasons

ADAS is especially vulnerable to malicious actions during updates because some interfaces which are not normally available become open to external data or external operations. Furthermore, the ADAS system consists of multiple modules. Any changes to any one of the modules will require the system to re-establish trust relationships between these modules so that they can reliably exchange data and commands. Lifecycle management security is not a new problem for the ADAS system. Any computing system needs to deal with changes to ensure that the system can “start secure—run secure—stay secure.” The same problem in the ADAS system is facing extra challenges: • Secure update on control systems immature in the auto industry and ecosystem: Manual update at a garage or repair shop by trained professionals is a common practice. The update process usually requires proprietary tools and labor-intensive work. Although there is a new trend of attempting to enable remote updates via standardized processes to relieve labor costs the procedure is still not mature enough to be pervasive. This is especially true for the updates that require intense verification of control system integrity. • Small-scale ECUs to meet security primitive requirements for secure update Updating relatively large-scale computing platforms is straightforward since there are available system update and recovery technologies and services suitable for such platforms. ADAS, however, may have smaller-scale microcontrollers for sensors and actuators. Such small-scale platforms may not have sufficient cryptographic or security primitive support. Hence, the challenge is to achieve the same security objectives with lightweight system update security technology. • Long lifetime vs. limited cryptographic strength Control systems like vehicles typically have long lifetimes. Cryptographic solutions in the current computing world, however, have relatively shorter lifetimes. Hence, during the ADAS system lifecycle, there may appear a need to update the ADAS system with a stronger and new cryptosystem. This problem in the traditional computing system is not critical given that most of the devices must be operational only for a few years. Careful design and analysis is required for updating the cryptographic system because it effectively serves as the basis of trust for every security function. Compromising the root of trust will surrender control to attackers.

ADAS Technology Challenges[9]

One factor that could influence ADAS uptake is the rate at which the technology advances. Although semiconductor companies and other players have made important enhancements in recent years, there is much room for improvement. A typical ADAS application incorporates many technologies, as shown in the figure below, but four stand out with regard to the challenges they present: processors, sensors, software algorithms, and mapping.

  • Processors. Electronic control units (ECUs) and microcontroller units (MCUs) are essential for most ADAS applications, including autonomous driving. For ADAS to advance, processors need better performance, which could be enabled by multicore architectures and higher frequencies, as well as lower power consumption requirements.
  • Sensors. These devices gather information on their immediate environment, such as pedestrians and oncoming cars. Most have a limited measurement range and signal bandwidth, which makes it difficult to distinguish between “signal” (for example, obstacles in the road) and system “noise.” It is especially difficult for sensors to track moving objects during less-than-ideal environmental conditions, such as rain and fog. Many industry players are trying to improve individual sensors. They are also attempting to optimize system performance through better sensor fusion—the coherent combination of data from multiple sensors. On the hardware side, intersensor communication is a major challenge because it requires high bandwidth and solutions for preventing network overloads. Players are currently optimizing the partitioning and distribution of system architecture to address this issue. On the software side, the fusion of image and nonimage data is particularly challenging. Some OEMs and tier-one suppliers are working together with academia to address this challenge, as can be seen in Daimler’s collaboration with the Karlsruhe Institute of Technology and the University of Ulm. The limited functionality of today’s sensors, combined with their high cost, maybe the greatest constraint to ADAS uptake. Many companies are making progress on both fronts, however. As one example, Mobileye and various start-ups are trying to improve the functionality of camera-based solutions, which typically have difficulty detecting obstacles during rainstorms or in other situations when visibility is limited. If camera-based solutions catch up to radar and lidar in functionality, they could eventually dominate the ADAS market because of their lower cost. “One box” solutions that combine lasers and cameras may also become popular because they are less expensive than radar or lidar alone. This is an important development since experts believe that semiautonomous driving will not become a reality until the industry has a cost-effective lidar system that is fully integrated with other sensors.
  • Software algorithms. Running on ECUs and MCUs, algorithms use the input from sensors to synthesize the environment surrounding a vehicle in real-time (going above and beyond the processing that sensors have already completed). The algorithms then provide output to the driver or specify how the system should actively intervene in vehicle control. This could require some of the most complex in-car-software integration ever created, since any decisions that the algorithms specify, such as the application of emergency brakes, are critical to ensuring safety. In response to developments in sensor fusion, the industry is about to transition from embedded software running on a single ADAS-specific ECU to software platforms running on centralized ECUs or MCUs. These software platforms have a higher level of abstraction to allow flexible integration of sensor-fusion algorithms. Industry players are now focusing on creating such algorithms, which allow for more accurate synthesis of sensor data and more efficient processing because they will help prevent data overload or slowdowns. Another priority is creating algorithms that allow for safer car navigation and more accurately predict all possible human behavior—including potentially irrational responses—in various situations, such as when a collision between two cars appears imminent.
  • Mapping. When GPS coverage fails, such as during tunnel travel, detailed and accurate mapping systems can help prevent accidents. These systems also store geographical and infrastructure information, make updates as needed, and communicate with onboard sensors to determine a car’s exact location. OEMs and other players in the automotive industry are looking for lower-cost methods to construct and maintain maps. Some of the most recent solutions include deploying “mapping cars” equipped with 3-D lasers and 360-degree high-definition cameras. Map developers are also leveraging data from sensors installed on commercial fleets, such as FedEx, as well as GPS data from drivers.

Advanced driver assistance systems 2
source: McKinsey&Company

Advanced Driver Assistance Systems will lead to rapidly increasing volumes of data within onboard networks: Additional sensors with various techniques will be necessary for the detection of the entire car environment. They will deliver data throughout any drive, especially video and cameras will account for large amounts of data. The collected data needs to be merged to get a truly reliable “picture” of what is going on around the car. Both developments require a high-performance computing capacity that the usual single ECU solutions won’t be able to provide in the future.[10]

Advanced Driver Assistance Systems (ADAS) and Automated Driving (AD): The Evolution[11]

Cars have come a long way since they first came on the road in the 19th century. The first semiconductor-based noteworthy assistance was a GPS-based car navigation system towards the end of the 80s. Today, in almost every car ABS (Anti-Blocking System) and ESP (Electronic Stability Program) are also contributing to the driver and car passenger’s safety using high-quality electronics. After all, the dreams of automated cars from the 60ies are now becoming true - it is only a matter of a few years. However, before automated driving, which avowedly is the ultimate goal of carmakers and consumers, assisted driving, which is currently experiencing a strong upwind will establish itself on a large scale.

Advanced driver assistance systems 1
source: Renesas

The paramount goal of ADAS is to assist the driver with the safety aspects – for himself and equally so for other mobile and pedestrian traffic participants. Next, to this, ADAS is supposed to enhance the driving comfort for the driver and finally yet importantly improve the economic as well as the environmental balance.

See Also

Industrial Control System (ICS)


Further Reading