Harman Advanced Concept Brings Intuitive HMI to Automated Driving

March 20, 2016

Ian Adcock

Time for another chapter before my destination? This future "driving" scenario is a glimpse into Harman's LIVS in the Etos concept. ZF TRW developed the restraint systems and steering 'wheel' that retracts into the fascia.

Harman International is demonstrating an advanced connected-car platform that previews a fundamental shift in the human-machine interface of automated and autonomous vehicles. For this project Harman is collaborating with Swiss-based concept vehicle specialist Rinspeed, which developed the BMW i8-based Etos demonstrator featuring V2X (vehicle to everything) capability and incorporating the latest Microsoft "driver assistant."

Rinspeed Etos interior shows design potential of the HMI.

The Harman platform, known as LIVS (Life-Enhancing Intelligent Vehicle Solution), is based on RoadLINK technology from NXP Semiconductors. The demonstrator shows how vehicle occupants will safely interact with the outside world, whether driving in SAE Level 2 or 3 automated mode, or in fully autonomous SAE Level 5.

Harman’s aspiration is to develop an “intelligent, intuitive system that, like an executive assistant, always reads what the boss wants, when you’re in a good or bad mood, under stress, not putting that irritating ‘phone call through or when you need a tea or coffee,” explained Phil Eyler, President, Harman Connected Car Division.

Microsoft assistant 'learns'

To demonstrate that potential at the 2016 Geneva Salon, Eyler ‘drove’ the author through an animation depicting a future commute in the Etos concept from home to the office.

“The key is moving from disparate technologies to more integrated ones in the overall experience,” he explained. Although the car is stationary during this virtual drive, it is always "live" and connected to "the Cloud" via Harman intelligent vehicle technologies. The first thing it does is authenticate the passenger and driver, via their mobile devices, as they approach. This in effect "personalizes" the vehicle to the occupants' individual tastes as Microsoft personal assistant "welcomes" the occupants to the car.

Sitting in the car, you're facing a pair of ultra-high-definition (4000 resolution), 21.5-in screens, “curved for the first time in a car,” noted Eyler. The screens can be individually configured to the driver and passengers preferences plus a smaller 2000-resolution center display. Both screens welcome driver and passenger whilst seamlessly integrating their home and car networks,

“Because it’s connected to 'the Cloud' and exchange server with Microsoft, the car has access to all their contacts, calendar, etc., and can 'see' that the next destination is his office,” Eyler explained.

As the vehicle emerges from the driveway it is already warning the driver of oncoming traffic in the blind spot: Etos is equipped with eight HD cameras to give the driver a surround view of the car, including objects in blind spots and ‘curb’ cameras for close proximity parking either in the street or in narrow bays.

However, unlike Continental's rear-view camera system shown last year with its trio of cameras (one each side and a third in the ‘bee-sting’ aerial mount’), the Harman system depends on two backward-facing cameras with the images displayed on the screens. Eyler admitted, however, that a third rearward facing camera could be installed and that the location of the display screens would be determined by the OEM customer.

Once en route, if the driver regularly asks the navigation system to guide him or her via a particular coffee shop, for instance, the Microsoft’s personal assistant will "learn" this and the personal assistant will, in the future, pre-empt the driver using voice activation rather than waiting for a request.

Depending on driving mode, the electronic 'assistant' HMI will stream music or play videos to each side of the cockpit. If the car is in SAE Level 5 full-autonomous mode with the steering wheel retracted into the fascia, the driver can enjoy a game on the embedded X-Box One.

Depending on the driving mode, the electronic "butler" will stream music via Harman’s intelligent player that categorizes music based on its genre, such as classical, jazz, etc., or play the driver or passenger’s favorite movie on their individual sides of the cockpit. If the car is in full autonomous mode with the steering wheel furled back into the fascia then the driver could enjoy a game on the embedded X-Box One.

If, however, the electronic assistant is aware of an impending conference call, for example, it will patch the call through complete with Power Point presentations.

Monitoring driver awareness

“Discounting the fully autonomous mode, we’re already working with OEMs to offer Microsoft in cars by the end of this year/early 2017, some as facelifts of existing platforms," Eyler said, "whilst we’re also developing with the display manufacturers to integrate them into our new generation of platforms.”

Integral to the car’s communication strategy is the technology's ability to determine if the driver is capable of receiving information under safety critical conditions. To this end the interior features five tracking cameras, including one that monitors the driver’s iris to measure cognitive load to understand the driver’s condition. There are also gaze tracking cameras measuring biometric rates, as well as mood detection to see if the driver is alert or needs prompting to rest and refresh themselves before continuing the journey.

Unlike some concepts this one studiously avoids the vexed "how" question when it comes to autonomous driving and driver/passenger safety.