The Vehicle as a Gaming Device

Audi spin-off Holoride uses VR to turn the back seat into an entertainment platform.

(Holoride)

From the moment I slipped the Oculus Rift device over my head, I was transported from a routine urban drive in the backseat of an Audi Q7 into an enchanting underwater world. The Virtual Reality (VR) experience obliterated every sight and sound of city life from my consciousness. My fellow backseat passenger and I were taking the demo during a break from the TechCrunch Sessions: Mobility 2019 conference last July. She squealed with laughter as our mini-submarine submerged into a seascape filled with schools of jellyfish, gently gliding Blue whales, and other undulating aquatic creatures.

The experience was provided by Holoride, a startup spun off from Audi in December 2018. The company performs a mind-bending game of pulling data from the vehicle’s inertial measurement unit (IMU), feeding it to processors in the VR headset. Throughout our 14-minute ride, the g-forces produced by the vehicle’s acceleration, steering, and braking were mimicked by the submersible I was “piloting.” The alignment of my real-world bodily motion and the movement of the virtual underwater vehicle produced an uncanny ultra-realistic effect.

Delivering ‘elastic content’

“The immersive-ness of this solution comes from actually feeling how the car moves. It’s not tricking you,” Nils Wollny, chief executive of Holoride (below), explained at the conference. “Suddenly, the car turns into a fantastic gaming device.” SAE’s Autonomous Vehicle Engineering was among the first to experience Holoride’s underwater ride, which debuted at CES Asia in Shanghai a month earlier. The 3D aquatic submarine content was created by Mackevision, an animation and visual effects studio based in Munich, Germany.

Holoride previously gave VR-enhanced rides at SpeedVegas, a racetrack in Las Vegas, during CES 2019. In that case, backseat passengers experienced a rollicking Marvel-based space battle produced by Disney Games and Interactive Experiences. My ride was tranquil by comparison. “This is the first time we are showcasing our technology in the city with real traffic,” said Daniel Profendiner, the company’s CTO. “We needed calmer content,” he said, speaking from the front seat. “Like a dive in the ocean where you can relax and learn something.”

The ability to vary the narrative is a core strategy for Holoride. Profendiner entered our driving route in downtown San Jose into the Q7’s navigation, allowing the VR console to anticipate turn-by-turn directions. “Our software procedurally generates the content while the car is on the go,” explained Wollny. “You can predict a lot about how you build the content and arrange it around the car.”

For example, as you approach an intersection, there are only four different ways you can turn. Those possibilities were anticipated by the game developer using Holoride’s software development kit (SDK). Thus, the game will guide you past asteroids or diving dolphins in one direction or another but never crash into them. The game also knows how long the trip will last so the narrative can create a satisfying conclusion – by speeding things up or slowing them down – before you reach your destination.

“It’s a completely new way of telling stories,” said Wollny. He said the company could eventually pull data from city infrastructure, like stoplights, to further coordinate the storyline. “We call it elastic content because it’s adapting to the environment.” The data set could eventually include the car’s rain sensor, thereby producing a storm in the virtual environment.

Moving theme park

Wollny knew from his first moment experiencing in-car VR in 2016 that he wanted to form a company to commercialize the technology. At the time, he was Audi’s head of digital business. His co-founders also worked at Audi. Profendiner was developing algorithms for self-driving cars, while Marcus Kühne, the chief information officer and third co-founder, worked on VR sales tools. Kühne is also the founder of Germany’s biggest VR community.

Audi allowed the three to spin off Holoride as a separate company in December 2018; company operations began February 2019. The automaker realized that success for the venture depends on turning Holoride into a platform used by all car brands. “We started Holoride with a vision. We want to turn every vehicle into a moving theme park,” said Wollny. Audi maintains a minority interest in Holoride through its Audi Electronics Venture subsidiary.

In San Jose, the experience was run on a gaming computer in the trunk. But Holoride will shift processing functions to the headset. Knowing that integration cycles in the auto industry can take years, the founders took an agnostic approach to headsets. In that way, the system will benefit from advantages in VR technology, such as faster frame rates. “The idea is for users to take a standalone headset you can buy on Amazon, hop into the car, and pair it as you do with a smartphone,” said Profendiner. Holoride is considering various business models, Wollny acknowledged, such as ride-hailing, that make dedicated VR devices available to users.

Holoride also plans to integrate an inside-out technical strategy in which the VR environment changes based on headset movements. That is not yet in place. While the company wants to add one-button controls for actions like shooting objects, it will refrain from using hand-based touch controllers in the vehicle. “You could smash other people and the windshield,” said Profendiner.

2021 market intro

The technology has proven helpful for reducing car sickness. Latency, already down to milliseconds, is not an issue. Car sickness occurs when the visual cues experienced by a rider are disconnected from balance functions in the inner ear. While Holoride is focusing on entertainment applications, it could also be helpful for productivity, allowing passengers to read or conduct meetings without feeling nauseous.

The company’s primary focus in the next year or so is to build a robust content-creation pipeline. Holoride is enhancing its SDK, a plug-in for game-creation engines such as Unity and Unreal. At the same time, the company is reaching out to major content studio and onboarding smaller production companies. Besides Mackevision, which created the underwater experience, Holoride also made its SDK available to VR studios Rewind in England and Schell Games in the U.S.

Aiming for market introduction in 2021, Holoride will entice studios with a business model that shares revenue generated when users download original content. The fees, which have not yet been determined, will be shared between the content creator and the automaker, a new revenue stream for OEMs to monetize data already generated by the vehicle.