HoloLens 2 Display: The Bigger Picture
Microsoft Advanced Optics GM, Zulfi Alam, discusses the
unique engineering advancements behind the larger field-of-view in HoloLens 2.
Listen in as Zulfi dives into the “largest small mirrors in the world,” to the
challenges created by lasers and waveguides, a combination of technologies that
naysayers said couldn’t be done.
Transcript provided by TheRealNiblicks
Transcript provided by TheRealNiblicks
Q: Hello everybody, I'm joined here with Zulfi Alam. Thank you. Hello.
Before we go and talk about the Hololens display, I hear you have a video to show us.
So lets go ahead and go to the video first:
<Video plays>
That was amazing. So, can you tell us who you are and what you do at MSFT and what you are all about?
Zulfi Alam: Thank you, Thank you, Thank you.
My name is Zulfi Alam. I’m the general manager for optics but before we get to my part of the presentation, I want to know what is up with the outfit?
Q: Well, you know, I though I would change things up a bit. Everyone’s dressed so nicely and everything. I thought I’ll be a bit different so I’m a giraffe today.
ZA: We should have coordinated.
But anyway
Thank you so much. But what I do want to do is talk about the display and the display tech that we have developed. When we talk about the display the first thing I want to address and talk about is the custom silicon. We are one of the unique companies on this planet that have our own custom silicon that can design things right from scratch. We have our own optics team. We have our own systems team. We have our own software team. And, we have our own algorithms team. There are not many companies in this planet.
Q: Right we have a lot of investment in this.
ZA: That can… Yes. that’s right but this is all under one umbrella so we can innovate in a really fast pace and come up with really novel solutions. So when we wanted to build this first genera…second generation display we were like the technology just didn’t exist and we had to develop it from ground up. So we developed our own custom silicon. We developed our own MEMS based display and we can get into why we went into this MEMS direction. But, we developed this display. We moved away from LEDs down to lasers. Much more light efficient.
Q: From Hololens1 to the next, the second Hololens.
ZA: Correct.
The first Hololens was LED based. We went to lasers. And then instead of using a LCOS or a DLP-type approach we went to these micro-electronic mirrors called MEMS. And essentially tiny mirrors moving back and forth really fast and essentially rendering the image. And the advantage of this is obvious: When you have a chip, as you think about increasing field of view, the chip just gets bigger and bigger. When you have this MEMS approach and as we think long term we can simply change the scan angle of these MEMS and essentially render a bigger display. We’ll talk more about that in a sec.
Q: OK.
ZA: But, essentially we went for this MEMS display. And the advantages are super crisp and super obvious.
The first thing is the field of view has dramatically jumped by 2x. We started off with 36 degrees. We are up to 51. That is twice the display. Same form factor. And or lighter. And or smaller. So, normally when you make things bigger you don’t stay the same or go smaller. So this is a huge huge accomplishment by the people working on this and they are the most amazing development team on the planet.
The next thing we did was comfort. Every human is unique. This is a wearable device. As you go about trying to design a device that doesn’t need 10 different SKU’s (Small/medium/Large/Male/Female) We were able to encompass all of that because we designed from the scratch and for these humans and we said, hey, this device is going to be the best in class enterprise device on the planet. As you saw, these are people on the development team and they had different heads and form factors and they are all able to wear the same device.
Q: Like I have a different head than you have a different head.
ZA: And that’s a point right. You don’t want to have people spend $3500 and just customize for one human. You want to pass the UI and enjoy the same experience. And um and finally is contrast: this device has amazing contrast because you’re based on these are we can switch the lasers off where the hologram is not important. So we have these two images side by side.
As you can see, and I’ll try to put my cursor on this, the hologram ends here but you can still see this haze. That is what happens when you have a LCOS based system. On the Hololens2 system, essentially the display system effectively switches off.
As you can see, and I’ll try to put my cursor on this, the hologram ends here but you can still see this haze. That is what happens when you have a LCOS based system. On the Hololens2 system, essentially the display system effectively switches off.
Q: Right so when there is nothing to show, the laser is off and you can see through. You can see straight through it. If I have a hologram I can behind it if there is nothing there, right?
ZA: Accurate. And that is the fundamental. It is 2500:1. That is the best in class. We are super proud of the work. So that is all I had in terms about talking about. If you had any questions.
Q: What I’m really curious about is maybe you can talk about how is this different than other similar devices. Like, how is this different than the Magic Leap display.
ZA: That is a good question. I want to be careful about how I answer this.
There are multiple great companies on this planet AKA Apple, Google, Magic Leap is one of them. They are attempting go off the same holy grail which is to build a head mounted device that is awesome. The approach that which we took is fundamentally different. Because, we said hey: We are going to make sure this device is comfortable for all users. So, we designed the eye box. The activator that you can see to be much much larger than anybody else can do. We have eye relief that is much larger than anybody else. And we are the only device that you can actually read text on. Imagine that you are an enterprise worker and you want to read a manual while you are trying to repair something. You can actually read the text. So, the reason why we are able to do all these so effectively is because we have the ability to simulate the production of a photon in the laser all the way through the light engine through the waveguide into your eyeballs. No one else has the ability to do that.
Q: How do you do that. How do you know where the eyeball is? I mean when I put the Hololens on it could be all kinds of directions. Right?
ZA: You're right. This is the amazingness of Microsoft algorithms. The whole point is you have no references. Your head can be anywhere and you want to put an image stable in front of you while your head is moving. Versus the other way around. So, we have these algorithms that I relate to projection that essentially know based on head movement where your head is going to be, and we project the image. We fire the laser off right at the right time to make sure we start the rendering of the image at the right place.
Q: Even if I have glasses on or no matter what I have in front of the middle of it. It still works?
ZA: That is correct. Because the eye relief is so much larger than anybody else we can accommodate glasses.
Q: But the difference between my eye and the screen, there is a lot of difference there right?
ZA: No problem we can accommodate any eye relief, not any eye relief, but we can accommodate eye relief that encompass 99.9% of humans including glasses.
Q: But the difference between my eye and the screen, there is a lot of difference there right?
ZA: No problem we can accommodate any eye relief, not any eye relief, but we can accommodate eye relief that encompass 99.9% of humans including glasses.
Q: That’s great. So lets talk about the field of view. You mentioned that its twice the field of view of the Hololens 1. How did you actually get to twice the size. How is that possible?
ZA: So, what we did was, instead of going with this LCOS approach where we need it for a larger field of view you needed a larger imager. We went the other way. We went with this MEMS approach. So, essentially changing scan angle we were able to produce an image that is large as the pixel pipeline can support. So, the pixel pipeline is designed to support 51 degrees. Our scan angle of emitters are able to support that. So we can increase the image size. Which is different from the original approach which is hey this if fixed 36 degrees.
ZA: So, what we did was, instead of going with this LCOS approach where we need it for a larger field of view you needed a larger imager. We went the other way. We went with this MEMS approach. So, essentially changing scan angle we were able to produce an image that is large as the pixel pipeline can support. So, the pixel pipeline is designed to support 51 degrees. Our scan angle of emitters are able to support that. So we can increase the image size. Which is different from the original approach which is hey this if fixed 36 degrees.
Q: So this is a whole new technology for the screen. You completely replaced the old technology with a whole new thing.
ZA: This is a whole new way.
ZA: This is a whole new way.
Q: Why did you decide to go this direction with the display? Like why did you decide with lasers? I mean lasers are cool of course. Other than that.
ZA: SIZE, WEIGHT, AND POWER.
Q: Right.
ZA: So, lasers are cool they are also the most efficient mechanism by which we can produce light. So, hence that was the right choice. It has its own set of challenges but it is the right call. Because of the MEMS approach, as we increase the field of the view the weight doesn’t change. So, it is also lighter than the original design point. And again the SRG’s, the waveguides, are, they are the best in class. So, we are able to maintain our size and power constraints and yet deliver a much larger field of view.
Q: That’s amazing. And so how did you actually make it so it fits multiple people. Multiple people can use it. What was the process that you went through to test this out. Make sure this works on these different.
ZA: Yeah, good question. We started with a database of like a publicly available database what are the head form factors. Then we built models in the house. Hundreds and hundreds of models and thousands and thousands of data points. We essentially scanned the heads of different humans. And then you come up with the spec say hey you what do you expect a human with what the eye is. Where is his eye going to be vs. where the lens has to be. And what is the maximum we can accommodate? So this is essentially a very tedious exercise of collecting thousands and thousands of head scans and then building a spec that supports all of them.
Q: So I bet you went through all sorts of Microsoft employees even outside of that. Kinda … everybody: hey just put this on, don’t worry what is right now.
ZA: I wish it was that straight forward but yes, we did build multiple… We actually built a setup just to measure heads of humans.
Q: Then you talked about high contrast. Could I use this outside? In a sunny day in the park…can I…How does it work.
ZA: So, previous devices have been sort of capped at very low number of nits. So 500 nits. This device yes you can.
Q: So I bet you went through all sorts of Microsoft employees even outside of that. Kinda … everybody: hey just put this on, don’t worry what is right now.
ZA: I wish it was that straight forward but yes, we did build multiple… We actually built a setup just to measure heads of humans.
Q: Then you talked about high contrast. Could I use this outside? In a sunny day in the park…can I…How does it work.
ZA: So, previous devices have been sort of capped at very low number of nits. So 500 nits. This device yes you can.
I’m not sure if we have committed to the number outside the company but we are designing this device so that it can go to the extremely high nits…over a thousand and you should be able to wear this in an outside environment.
Q: And then how do you manage to use the lasers to actually display the image two-dimensionally. Like right now you have one laser…you have mirrors. How do the mirrors work together with the laser. And how does that work?
ZA: Essentially you have a fast scanner that is essentially scanning in the horizontal direction. And then you have a slow scanner that … once you paint the image horizontally…you move it one pixel down, you start painting it horizontally. Two mirrors working in conjunction with each other. One working on the horizontal axis. One working on the vertical axis.
Q: And the resolution of this screen is really large right? How fast does this actually mirror actually scan through it, right?
ZA: 54,000 times a second.
Q: So 54, 000 times you have a laser that…
ZA: That is the mirror cycle time. It is 54000 times a second. And each pixel you are firing..the laser if firing for each pixel. So it is, like yeah.. it is like a couple of million pixels that we are able to render. So yes. And the text readability of this device is amazing. So, our internal metric is essentially 8 point font. Developers should be able to make content that allows them to render font that is 8 point font. Which is pretty cool.
Q: So 54, 000 times you have a laser that…
ZA: That is the mirror cycle time. It is 54000 times a second. And each pixel you are firing..the laser if firing for each pixel. So it is, like yeah.. it is like a couple of million pixels that we are able to render. So yes. And the text readability of this device is amazing. So, our internal metric is essentially 8 point font. Developers should be able to make content that allows them to render font that is 8 point font. Which is pretty cool.
Q: That is incredible. Thank you so much for being here with us. I learned so much. I hope everyone else learned a lot.
ZA: Thank you so much for having it.
Comments
Post a Comment