The Verge: Hololens 2 Hands On

MICROSOFT’S HOLOLENS 2: A $3,500 MIXED REALITY HEADSET FOR THE FACTORY, NOT THE LIVING ROOM

The biggest complaint about the first HoloLens was simple: you only saw the holograms in a relatively small box, directly in front of you. Turn your head even a little, and they would disappear from your field of view. Worse, their edges would clip out of existence even when you were staring right at them. It was like looking at a digital world through a tiny rectangle.

The HoloLens 2 has a field of view that’s twice as big as before. It doesn’t quite fill your entire field of vision — there’s still clipping — but it’s big enough now that you no longer feel constantly annoyed by a letterbox. Microsoft says that each eye has the equivalent of a 2K display in front of it, but it’s better to think of that as a metaphor than a precise spec. The exact spec is that it has a “holographic density of 47 pixels per degree,” which means that the pixel density is high enough to allow you to read 8-point font.

Typically, when a tech product gets better specs like these, it happens through sheer force of technical iteration: faster processors, bigger batteries, more RAM, and so on. But that strategy wouldn’t have worked for the display on the HoloLens 2. It needed to get lighter, not heavier. So Microsoft had to completely change over to a different kind of display technology.

LASERS AND MIRRORS

Laser-based displays have become the thing to do for computers on your face. Intel’s Vaunt project used lasers, and the North Focals smart glasses do, too. Although Microsoft is using some of the same basic components, it’s taken them in a different direction and gone much further in developing what they can do.

The lasers in the HoloLens 2 shine into a set of mirrors that oscillate as quickly as 54,000 cycles per second so the reflected light can paint a display. Those two pieces together form the basis of a microelectromechanical system (MEMS) display. That’s all tricky to make, but the really tricky part for a MEMS display is getting the image that it paints into your eyeball.

One solution that companies like North have used is a holographic film on the lens to reflect the image directly into your retina. That has lots of drawbacks: a tiny display and low resolution, for two. But the truly problematic part is simply ensuring the display is aimed right into your eye. You have to be custom-fitted for the North glasses, and the image can disappear entirely if they’re misaligned.

Microsoft doesn’t want any of those problems, so it turned to the same thing it used on the first HoloLens: waveguides. They’re the pieces of glass in front of your eye that are carefully etched so they can reflect the holograms in front of your eyes. The waveguides on the HoloLens 2 are lighter now because Microsoft is using two sandwiched glass plates instead of three.

When you put the whole system together — the lasers, the mirrors, and the waveguide — you can get a brighter display with a wider field of view that doesn’t have to be precisely aimed into your eyes to work. Zulfi Alam, general manager for Optics Engineering at Microsoft, contends that Microsoft is way out ahead with this system and that waveguides are definitely the way to go for mixed reality. “There’s no competition for the next two or three years that can come close this level of fidelity in the waveguides,” he argues.

Do you want a wider field of view? Simple. Just increase the angle of the mirrors that reflect the laser light. A wider angle means a bigger image.

Do you want brighter images? Simple again. Lasers, not to put too fine a point on it, have light to spare. Of course, you have to deal with the fact that waveguides lose a ton of light, but the displays I saw were set to 500 nits and looked plenty bright to me. Microsoft thinks it could go much brighter in the final version, depending on the power draw.

Do you want to see the holograms without getting specifically fitted for your headset? Simple yet again. The waveguide doesn’t require specific fitting or measurement. You can just put the headset on and get going. It also can sit far enough in front of your eyes to allow you to wear whatever glasses you need comfortably.

Simple, simple, simple, right? In truth, it’s devilishly complex. Microsoft had to create an entirely new etching system for the waveguides. It had to figure out how to direct light to the right place in the waveguides nearly photon by photon. “We are simulating every photon that comes from the laser,” Alam says. The light from the lasers isn’t just reflected; it’s split apart in multiple colors and through multiple “pupils” in the display system and then “reconstituted” into the right spot on the waveguides. “Each photon is calculated where it’s expected to go,” Alam says. That takes a ton of computing power, so Microsoft had to develop custom silicon to do all of the calculations on where the photos would go.

And though alignment is much easier with the waveguide, that doesn’t mean it’s perfect. That’s why there are two tiny cameras on the nose bridge, directed at your eyeballs. They will allow the HoloLens 2 to automatically measure the distance between your pupils and adjust the image accordingly. Those cameras will also allow the HoloLens 2 to vertically adjust the image if it gets tilted or if your eyes are not perfectly even. (They are not. Sorry.)

A sort of free benefit of those cameras is that they can also scan your retinas to log you into the HoloLens 2 securely. It runs Windows, after all, and therefore it supports Windows Hello. They also track where you’re looking, which enables some new user interactions I’ll get to below.

A MEMS mirror under a high-speed camera.

 GIF: Microsoft

Then there’s power: lasers, oscillating mirrors, and custom chips to handle the computing for all of that must chew through battery. But Alam tells me that even with all of that, it still manages to require less power than the alternative. The mirrors oscillate in resonance, so it takes less energy to move them, sort of like they’re the fastest metronomes ever. Lasers are also less lossy than LEDs, and custom silicon can be optimized to its specific task.

”Our evolution is toward a form factor that is truly glasses,” Alam says, “and all these are significant steps in this journey.”