AR/VR/MR 2020: The Future Now Arriving

06 February 2020

 

By Rebecca Pool

"We've had Palmer Luckey, the 19 year old that put together the Oculus Rift VR headset with duct tape and sold his company to Facebook for $2 billion, and we've got Magic Leap, a recent MR start-up that's worth more than $7 billion," highlights Bernard Kress, Principal Optical Architect on Microsoft's Hololens MR Project, and also instrumental to the design of Google Glass. "The investment in augmented and virtual reality since the Google Glass days has been just amazing, and this has generated a lot of excitement."

Best of sensors
In addition to light engines and optical combiners, sensors - from depth mapping to head tracking, gaze tracking and gesture sensors - are critical to the XR experience. As Kress puts it: "You can have the best display in the world but if your sensors aren't up to the task, then the experience will be a joke."

Kress and colleagues will be addressing the challenges in a series of workshops and courses from Sunday through to Thursday at SPIE VR/AR/MR. But as the Hololens Optical Architect highlights, developing low-latency sensors is critical as data needs to be rendered in real time to produce a convincingly smooth and responsive AR experience. "When you move your head, your display has to be compensated," he says. "The motion-to-photon latency needs to be less than 10 milliseconds - any greater and you'll be uncomfortable and nauseous."

With this in mind, Microsoft built a custom chip holographic processing unit for Hololens to swiftly process data from sensors and handle tasks such as spatial mapping and gesture recognition. "We're down to 9 ms with Hololens 2 and hope to be close to 5 ms with next versions - but much work still needs to be done to reduce that motion-to-photon latency."