About five or six years ago, a friend lent me a first-generation HoloLens, and the experience is still unforgettable.

HoloLens：

HoloLens is a mixed reality headset developed by Microsoft. The glasses track your movement and gaze, generating appropriate virtual objects projected into your eyes. Because the device knows your position, you can interact with virtual 3D objects through gestures, such as raising your hand in the air and clicking down.

I remember when I started RoboRaid, before I could react, bizarre alien robots broke through my walls and attacked me. Following the game’s audio prompts, I constantly changed my attack direction and then snapped my fingers like Thanos.

HoloLens game experience | Author’s photo

These robots were not very well modeled, but the emphasis was on “realism”. I clearly remember that after the game ended, I went to touch the wall to check if it really had holes blown in it. Even the gameplay videos from a few years ago still seem impressive today.

Compared to the Google Glass at that time, which had a virtual screen for display, HoloLens creates a 3D holographic image integrated into the real environment, and its binocular imaging provides a higher degree of integration than a monocular (Google Glass’s 2D image). The game scene is generated based on the real environment, “real” enough to create depth perception, occlusion, and interaction between people and the scene. However, the domestic price once soared to 70,000, which completely cut off my thoughts.

Two years later, I accidentally saw a “broken machine” HoloLens on Xianyu, priced around 400 yuan. A lightbulb went off in my head, and I bought one back, ready to fix it.

Fortunately, upon disassembly, I found that the core components (CPU, memory chips, and several main chips) were undamaged, and the internal damage was mainly to the circuit, which could be repaired with flying wires (a method of directly connecting two breakpoints on the circuit board with wire).

Unfortunately, the lens was broken, but since HoloLens’s optical lens is modular, I only needed to replace the optical lens (yes, I bought another HoloLens with intact lenses, also around 400 yuan).

Screen and “modular” optical module | Author’s photo

At first, I thought just installing the lens would be enough, but the first time I put it on, the image was blurry. I took it down, and the second time I put it on, one eye’s image was higher than the other. I kept adjusting and calibrating, and the good lens got damaged from repeated disassembly, and I almost gave up.

Repaired HoloLens, without the shell | Author’s photo

Fortunately, at the moment it powered on, the feeling of playing HoloLens returned. More importantly, during the repair process, I gained some understanding of the imaging mechanism of the waveguide lens, which laid a foundation for later.

Smart glasses are generally regarded as the “next generation computing platform” and should serve as tools to enhance efficiency, meeting daily needs such as translation and navigation (this is also the direction many manufacturers have been striving for recently). “Daily” means they cannot be too large or cumbersome, which is also my original intention for making smart glasses.

An AR device consists mainly of two parts: the processing chip and the optical module. The latter often distinguishes different AR glasses products. The optical module structure of AR glasses on the market is mainly divided into three parts. To put it bluntly, they serve to project images, magnify images, and reflect images.

Image projection refers to needing a screen body for imaging. Magnifying refers to enlarging the image size. Image reflection means using different optical technologies (such as Google’s prism, HoloLens’s waveguide lens) to reflect, refract, or diffract light, ultimately forming an image on the retina.

For example, Google Glass images are formed by projecting onto a small prism with a cut reflective surface. Waveguides have made significant improvements in size and field of view. To improve “optical efficiency”, minimizing loss during light transmission is key; “total internal reflection” means that light moves forward through back-and-forth reflections (like a snake) without being transmitted out.

Different optical technology principles | Huachuang Securities

For my first attempt, I tried to make a device using the waveguide lens disassembled from HoloLens. Additionally, the other two devices that meet the criteria are likely to be common projectors. Later, I bought a ready-made mini projector, using a Raspberry Pi as the processing terminal to create the first prototype. The downside is that the projector is large and power-hungry, failing to meet the needs of a mobile device.

The first version I made was very large | Author’s photo

Given my personal capabilities at the time, making a small enough projection optical machine was unrealistic. However, my thinking gradually became clearer — to control the manufacturing difficulty, I chose “raw materials” that cover the above three functions as much as possible. The waveguide lens of HoloLens, like traditional optical devices, only serves to “change the direction of light propagation”.

Thus, I abandoned the idea of using HoloLens lenses to make one and opted for more suitable accessories that could reduce the size.

I found a broken “Epson BT200” optical component (the old rule — disassemble!). Its image reflection structure and optical magnification structure are combined, and modification only requires changing the internal imaging (to put it bluntly, changing the screen). The BT200 uses free-form optical technology, avoiding the complex optical machine structure used by waveguides, making it relatively easy to modify.

I removed the BT200 screen, replaced it with an HDMI (input signal imaging) OLED screen, and connected it to the Raspberry Pi terminal to create the first “smart glasses”.

BT200 version of the glasses, later I added a 3.5mm headphone jack | Author’s photo

On my third attempt, I planned to develop a better version and spent 4500 yuan to buy a commercial array waveguide module. In the following days, I tried to watch “Planet Earth” and play “Animal Crossing”… By connecting via HDMI, I could “project” the game screen in front of my eyes, playing while sitting or lying down, no longer worried about the NS hitting my face.

Overall, the picture quality satisfied me. Then I wondered, beyond a simple screen, could I achieve more?

I added USB as an expansion port, compatible with the modules designed later. Based on the Raspberry Pi Linux system, I wrote some applications corresponding to different external modules. Reality problems quickly arose. Initially, I hadn’t thought through what the final form of the glasses would be, and when I wanted to add or subtract features midway, it became difficult. More functions meant more components and interconnections, and the glasses themselves had little space to accommodate them.

As a result, in the end, I only created an application for “recognition detection (of living beings)”, but the Raspberry Pi I used at the time only had USB2.0 ports, leading to insufficient power supply for the accessories, and the device couldn’t support the detector’s head. The idea of extending the glasses’ functionality almost failed.

But I didn’t want to interrupt this process. Since I couldn’t make a “Super Saiyan”, I prepared to separate the “recognition detection” function and make it into a “sight device”. (This function faced many issues on the glasses, making it impossible to improve further. It involves subsequent patent applications for the “sight device”, so I won’t elaborate.)

Changed to a sight device | Author’s photo

In simple terms, the “sight device” enhances the environment based on thermal imaging combined with image algorithms, possessing certain night vision capabilities. The thermal imaging function can penetrate smoke and detect objects with specific heat sources (useful for quickly searching for living targets).

I initially used a regular camera, but visible light cameras perform poorly in low-light environments. Adding extra lighting contradicts the goals of portability and low power consumption. Later, I tried thermal imaging, which does not rely on visible light imaging and has certain resistance to interference, as strong light flashes have no effect on thermal imaging.

One reason for making it a sight device is that I am also a half-military enthusiast. The basic design follows the previous glasses’ concept, aiming to find solutions for the many issues exposed by the glasses version.

The biggest issue is power consumption. The improved glasses already have significantly lower power consumption compared to the original “HoloLens version”, but the Raspberry Pi’s technology is average, and power consumption control isn’t excellent. Using a built-in battery, 2000 mAh is expected to last over 30 minutes. Can you imagine a “little gadget” hanging a 10,000 to 20,000 mAh power bank?

Secondly, the high temperature caused by the Raspberry Pi affects the experience of wearing it for long periods. Additionally, for this improved (sight device) version, I personally do not need video or multimedia functions, so reducing power consumption and heat generation is the primary goal. Therefore, I replaced the original Raspberry Pi-based driving scheme with a low-power STM32 scheme and upgraded the accessory performance, increasing continuous running time to about 7 hours.

During the pandemic, I didn’t go out often, so I used the sight device to look out the window and “detect” stray cats. To clarify, long-wave thermal imaging cannot see gases, so I can’t see whether passing pedestrians are farting.

Piercing smoke experiment | Author’s photo

AR is not a “specific device”, but a technology that enhances the real environment. HoloLens’s “environment scanning” is based on SLAM (Simultaneous Localization and Mapping), scanning the environment and overlaying the results with reality. This machine currently cannot achieve that.

But it has begun to fulfill my initial “fantasy” of AR glasses — interacting with the environment.

Although I tried to find solutions to the problems of AR glasses, the iteration has stagnated. The reason is that the production cost far exceeds the DIY scope; at that time, a waveguide lens cost 4,500 yuan, and I spent over 10,000 yuan changing accessories.

Manufacturers developing AR glasses also expose some difficult problems. The first is standby duration. Viewing AR glasses as the next generation computing platform, the standby duration is far from that of a mobile phone. Standby duration equals usage value. The second is screen brightness; currently, only monochrome screens can be used outdoors, while color screens cannot.

When OPPO Air Glass was released, some might ask why it wasn’t in color? The answer is this. Red and blue brightness cannot be achieved, while green has the lowest power consumption under the same brightness among the “three colors”. Moreover, color data requires more memory and consumes more power. Due to the optical technology solution used by OPPO, the content displayed on the lens can be seen clearly by others.

OPPO Air Glass display effect | Image from the internet

This reminds me that although Google Glass also had a color screen before, it solved power consumption and standby issues with low resolution (640*480), narrow field of view, and small screen solutions. Even so, continuous usage time did not exceed four hours. In contrast, while HoloLens is impressive (the image size matches that of a television), achieving high brightness for a large image requires compensating with power consumption.

I personally believe that Google thought carefully about making an AR glasses, as it is a product that integrates various factors. To some extent, it made me look forward to the concept machine at this Google I/O.

So, how far are we from “consumer-grade AR”? Perhaps it depends on whether manufacturers pursue HoloLens-like “coolness” or create a product similar to “Google Glass”.

People who click “like” have become more attractive!