Lei Feng network (search "Lei Feng network" public number concerned) by: author of this article Xie Hui, co-founder of Stanford Technology & CTO. Xiaolong (formerly Longlong) Technology focuses on smart glasses products and augmented reality technology research and development. Its XLOONG sports smart glasses are known as "Google Glass" in the sports field.
In January 2015, Microsoft released Hololens, its in-house developed holographic smart glasses, which are very different from the virtual reality glasses and augmented reality glasses on the market. They can identify the surrounding environment and the user's movements through See- Through the entire waveguide display program, to give users a holographic visual experience. Hololens was booked by developers in the first quarter of 2016 and shipped in the second quarter. The price was $3,000.
Leaving aside mysterious black technology such as Magic Leap, Hololens can be said to be the best AR and MR device for the current experience. From its name you can see Microsoft's ambition, Holo is the meaning of the entire system, and lens is The meaning of the lens, Microsoft does not follow the Google glass, called Microsoft glass or Hologlass, because Glasses is a frame of glasses, Contact lens is a contact lens, so that the lens is much larger than the glass. With the development of electro-optical technology, the display lens may be the same as the contact lens, so the pattern of Hololens is much larger than Google Glass.
Hololens hardware configuration 
Hololens full range of smart glasses and adapters, Micro USB data cable, spare nose pads, Clicker, brochures, glasses cloth and other accessories.
The Hololens hardware platform uses an Intel X86 32-bit processor. In conjunction with this processor, there is also a full-line processing unit (HPU) dedicated to holographic imaging and environmental awareness. It features 2GB of RAM and 64GB of ROM, and supports Bluetooth and WiFi.
As an extension of artificial intelligence, to understand the environment and people's intentions, HoloLens must also have powerful sensors to support it. There are a total of 4 environment-aware cameras in the left front and right front of the glasses. There is a center in front of the glasses. Ordinary RGB camera and a depth camera, these cameras mainly use infrared technology for gesture recognition and real-time modeling of the environment scene.
Hololens is also equipped with 4 high-sensitivity microphones that can pick up different angles of environmental sounds and voice commands. In addition, the inertial sensors, gyroscopes, and ambient light sensors on smart phones are also available, and inertial sensors and gyroscopes are mainly used to recognize the posture of the human head. There are a pair of stereo speakers at the ear positions on both sides of the glasses. The system controls the two speakers through an algorithm to simulate 3D sound effects. In addition, Clicker widgets can be connected to Hololens via Bluetooth, instead of gestures to interact with the glasses.
Hololens needs to achieve full-line display and environment awareness, gesture recognition and other functions, is bound to need strong hardware performance, high-performance will inevitably lead to high power consumption, Microsoft officials did not disclose Hololens machine power consumption and battery capacity, but according to the actual experience It can be used for about 3 hours, which is enough for an experience-based product. According to the author's estimation, the power consumption of Hololens should be 6-8W, and the battery capacity should be around 5000-6000mAh.
Hololens weighs around 550g (more than 1 kg) and it's hard to imagine what it would feel like to wear a 1 kilos of eyeglasses on the head, but Hololens' design is clever, with a ring-shaped head frame on the inside of the glasses. When worn, Adjust the head frame to a suitable size so that it presses on the top of the forehead, which can greatly reduce the pressure on the nose pads.
When it comes to full-line displays, Microsoft's optical display solution has to be mentioned. Hololens uses a holographic waveguide display technology. Its optical lens thickness is 3-4mm, and the field of view angle is about 30 degrees.
Compared with the off-axis reflection technology used by Meta2, the prism reflection technology used in this display technology has advantages of small thickness, small size, true color reproduction, and the like, and can be easily made into the appearance of ordinary glasses.
Corresponding to the holographic waveguide display technology, there is also a geometric waveguide display technology. The geometric waveguide lens can be made thinner. The current technology can achieve a thickness of 1.7 mm, and the viewing angle can be 60 degrees. However, the biggest drawback of these two kinds of waveguide technology is that the processing technology and processing are difficult, and the mass productivity is not very good, so the cost is relatively high. The author guessed that the cost of Hololens' optical lens may account for 1/3 of the whole machine.
Hololens software system
Hololens is equipped with a complete operating system based on the Windows 10 system, called Windows Holographic.
The system interface is also a Metro style interface that continues Windows 10. The system design is based on the interaction of glasses. Holographic's internal holographic application is called Hologram. Developers can use Holographic's tools to create holographic 3D models in Holostudio.
At present, Microsoft has announced that it will open the system to Intel, Qualcomm and other companies. This is similar to Microsoft's promotion of the Windows operating system in the PC field, the first to build systems and ecosystems, and then let other vendors and developers join their own camp to dominate the market , Microsoft With experience on Windows PCs and Windows Mobile, it is now well-known in AR and MR.
Prior to developing the Hololens app, developers needed to install Visual Studio 2015 or later, the Windows 10 SDK, the Hololens emulator, and the Unity Hololens technology preview. These tools can be downloaded from the Microsoft website. At present, there are already detailed development guidance documents on Microsoft's official website, and developers can understand it.
Hololens has a built-in parkour called Young Conker. When you click to start the game, HoloLens will first prompt you to first spatical mapping, that is, you need to scan your room first and you will see where you scanned. There will be a layer of crack-like blue nets. When HoloLens recognizes the entire room range and some specific objects in the room, it begins to enter the game. You can control the game fox by watching the focus move. In the trajectory of the game, the little fox in the game will “snap†with you in your room.
Hololens interactive experience
As mentioned earlier, Hololens integrates a depth camera and multiple sensors. After the CPU obtains the sensor data, Sensor fusion technology can be used to implement multiple interaction modes.
Hololens supports gestures, gaze, speech, and other interactive methods.
Currently, Hololens supports the following gesture interactions :
Index finger click for confirmation;
The index finger and thumb pinch and drag;
The gesture of blooming, that is, the palm of your hand is facing upwards, the five fingers are folded, and then opened again. This gesture is to open the start menu.
Gaze interaction , not eye tracking, can not move the cursor through the movement of the eye, the eyes need to stare at the cursor and move the head in order to move the cursor. By using the above two methods, most of the interaction requirements can be met.
Voice interaction is somewhat similar to Apple's Siri, but it's not much to talk about here. In addition, there is the Bluetooth remote controller Clicker mentioned above.
Hololens's head posture tracking mainly depends on gyroscopes and inertial sensors , because the head posture is mainly divided into XYZ three-dimensional displacements and angles. This technology is similar to that used on current smartphones, such as racing on mobile phones. The game needs to track the gesture of the phone.
Hololens's gesture recognition mainly depends on the depth camera. Similar to Microsoft's own Kinect on the Xbox, currently the technology of using depth information to obtain gesture data mainly includes TOF, structured light technology, and binocular cameras.
According to the author's understanding, Hololens uses the TOF technique, which obtains the information of the Z axis through the principle of infrared distance measurement. Finally, it obtains a series of depth maps, and then calculates the specific gestures through image processing. Because the depth camera uses infrared light, the accuracy of gesture recognition is relatively high during indoor experience. However, in the outdoor sunlight, the recognition effect will be discounted because the infrared component in the sunlight will cause a certain degree of depth to the camera. influences.
Hololens also has environmental awareness. When using some applications, the four IR cameras on both sides will scan the environment to build a 3D model of the external environment.
This technology is called SLAM technology, namely real-time positioning and map building technology. SLAM technology has been used in robots and unmanned aerial vehicles before, because these two types of intelligent hardware need real-time control of their position in the environment. The development of this technology It will also promote the development of VR, AR and MR.
Some applications in Hololens before the experience, will guide the user to turn the head to scan the room information, IR camera by measuring the depth of information in all directions, in order to generate a 3D model of the room, in the course of the game, the 3D effect will appear and the room The actual structure is integrated so as to achieve the effect of MR.
Hologram has a very important keyword - "anchor", such as an open astronaut model, as if anchored in the environment space, the astronaut model will not move with the movement of people, so people can be in space It is observed at 360 degrees, giving a holographic feel. These experiences benefit from SLAM technology and Microsoft's HPU.
Some people say that Hololens is a major move by Microsoft to move the computing platform from the PC side and the mobile side to the glasses side, and is another major technological revolution following the iPhone.
Such an assertion does not make comments for the time being, but from the perspective of the development of modern science and technology, personal computers, feature phones, and smart phones... The development of consumer electronics is designed to meet the needs of humanity for more efficient life, entertainment, and office work. The innovations are accompanied by lower costs, more natural interactions, and better experiences.
From this point of view, AR and MR are in line with the law of historical development. In the near future, they will certainly replace personal computers and smart phones, becoming the next generation of computing platforms. The layout of Microsoft and other giants in this field will inevitably speed up the process of this node. arrival.