Umlaut Benchmarks Vehicle Connectivity: Who’s on Top?

A 2021 study by global engineering consultancy Umlaut evaluated and rated connectivity and UX features of five luxury SUVs.

Seamless connectivity will continue to play a larger role as vehicles become more automated. (Cadillac)

Connectivity’s role in today’s vehicles is mostly focused on driver information and entertainment. But seamless connectivity will play a vital role in highly automated, and autonomous, driving. Autonomous vehicles (AV) will increasingly depend on the human machine interface (HMI) and the connectivity ecosystem around it. If, for example, the driver is video conferencing, checking a home camera or making music selections, even connectivity aspects which are not directly related to autonomous driving functions will impact both the user experience (UX) and the overall safety.

According to Umlaut’s Hakan Ekmen, some OEMs are well positioned for the connected and autonomous future of driving. (Umlaut)

Where do passenger-vehicle OEMs stand as of early 2021 in providing seamless connectivity? And, how competitive can they be tomorrow? A benchmark study  recently concluded by the global engineering consulting company Umlaut, offers insights into the industry’s connectivity progress. Umlaut, headquartered in Germany, formerly P3 Group, is known for its benchmarking across many industry sectors, including energy, telecom, aerospace, medical and logistics, in addition to automotive and others.

Umlaut’s Connectivity benchmarking activity focused on five luxury SUVs, selected for their market and technology relevance as well as the availability of connectivity-based features. The vehicles were the 2020 BMW X5, 2021 Cadillac XT6, 2020 Lexus RX450h, 2020 Lincoln Navigator and 2020 Mercedes-Benz GLE 450.

Perceived usability and UX

A team of experts assessed the features and UX of the tested vehicles’ connected functions and services. As a part of their evaluation, pre¬defined use cases were per¬formed independently by the testers. The assessment is largely based on the availability of features, which contributes a maximum of 700 points to the total score maximum of 1,000 points.

The Feature Availability test itself consists of three categories: Infotainment, Navigation and Connectivity. The remaining maximum of 300 points is awarded for the UX score, which is composed of the aspects Feature Performance (max. 240 points) and Feature Experience (max. 60 points). The determination consists of a total of 60 test cases from categories such as app/smartphone, navigation, entertainment, communication, voice control, comfort, productivity and extras. The finalists and winner, in ascending rank order:

5th place: 2020 Lexus RX450h

The fifth-rank vehicle, the Lexus RX450h F-Sport (base price $59,845), features driver-assistance suite that includes a lane tracking assistant; a pre-collision system with pedestrian detection; adaptive cruise control; a lane departure alert with steering assistant; a blind spot monitor, parking assistant and rear cross-traffic automated braking. In terms of connectivity, the Lexus features a nice ‘driver score’ option and notification if a guest user is exceeding driving restrictions. These highlights are balanced by shortcomings such as no wireless functionality of the remote user interface and no notifications on the smartphone for a theft alarm. In Umlaut’s assessment, the Lexus achieves 772 out of 1,000 points—23 points out of fourth place.

5th Place: 2020 Lexus RX450h (Lexus)

4th place: 2021 Cadillac XT6

The fourth-ranking Cadillac XT6 Premium Luxury ($83,540) features a comparably small 8-in. touchscreen. Regarding automated driving features, the vehicle offers cruise control; a forward collision alert; front pedestrian braking; lane change alert with side bind zone alert; a lane-keep assistant and a rear cross-traffic alert. In terms of connectivity, basic features such as onboard Wi-Fi, USB as well as wireless Apple CarPlay and Android Auto integration plus wireless smartphone charging are present, but Umlaut’s testers see some areas for improvement, such as more accurate navigation positioning and a more distinct reminder of theft alarm notifications. The SUV achieves a score of 795 points.

4th place: 2021 Cadillac XT6 (Cadillac)

3rd place: 2020 Lincoln Navigator

With a total score of 818 points, and thus 28 points behind the winner, the Lincoln Navigator 4x4 Reserve ($90,395) ranks third. The largest SUV ever manufactured by Ford features adaptive cruise control with stop-and-go, an active parking assistant, blind spot detection and a pre-collision assist feature. Its HMI combines a 12-in. digital instrument cluster and a 10-in. infotainment touchscreen, supported by a head-up display (HUD). The AppLink feature enables the projection and operation of favorite mobile apps on its infotainment screen. Lincoln’s PaaK (Phone as a Key) function allows vehicle operations without physically possessing the key. The biggest drawback in the Navigator found by Umlaut’s testers is that 911 emergency-call support is not available if the user’s smartphone is not connected to the head unit.

3rd place: 2020 Lincoln Navigator (Lincoln)

2nd place: 2020 BMW X5

The 2020 BMW X5 M50i ($89,445) ranks second, fifteen points behind the winner. The test vehicle, which scored 830 points, was equipped with an “active driving assistant” including frontal collision warning, lane departure warning, active blind spot detection and rear cross traffic alert. Like the Mercedes-Benz, the BMW also offers a 12.3-in. digital instrument cluster along a 12.3-in. infotainment touchscreen. Voice commands support calls, infotainment, route guidance or climate control. With this comfortable on-board operation, BMW’s “Connected” app provides control over numerous remote functions as well as a useful calendar integration.

In addition, the X5 offers gesture control as well as a digital key on Android phones. BMW supports Car Play or Android Auto only wirelessly and not via USB. iPhone users are forced to do without the digital key. Among the X5’s relatively few shortcomings reported by the testers was the fact that push notifications could not be received by the app when the phone was locked or while the app was running in the background.

2nd place: 2020 BMW X5 (BMW)

1st place: 2020 Mercedes GLE 450

The overall winner in Umlaut’s benchmark is the 2020 Mercedes-Benz GLE 450 ($67,195), scoring 846 points. In terms of highly automated driving features, the tested vehicle was equipped with a collision prevention system, blind spot detection and a parking assistance package. Regarding user experience, Umlaut’s testers praised the particularly future-oriented “MBUX” cockpit design, featuring two 12.3-in. displays. One of them serves as a digital instrument cluster and the other one as an infotainment touchscreen.

1st place: 2020 Mercedes GLE 450 (Mercedes-Benz)

The GLE offers a large list of convenience features, including a voice assistant which is invoked by the key phrase “Hey Mercedes”. However, the testers also identified some potential for improvements. For example, the search function for POIs or addresses could be more advanced and easier to access. Also, the manufacturer should provide wireless connectivity for the smartphone integration via Apple CarPlay or Android Auto. Still, the Mercedes SUV offered the best combination of Feature Availability and User Experience, earning it Umlaut’s “Best-in-Test 2021” accolade.


Drivers no longer want to go without internet access and the services connected to it. Connectivity is an important premise for autonomous driving, but at the same time it represents a big challenge for vehicle OEMs. The results of the 2021 Connected Vehicles Benchmark show that some automakers are already well positioned for the connected and autonomous future of driving, while others clearly have some catching up to do.

Hakan Ekmen is U.S. managing director of Umlaut.