Report from Xingkun at Aofeisi Quantum Bit | Official Account QbitAI

With cool gestures, control the electronic imaging projected in front of you, isn’t this just the basic configuration in sci-fi movies?

Now, someone has brought it from science fiction movies into reality. Move your fingers to gain control over the world in front of you.

YouTube creator Teemu Laurila, who is keen on making smart gadgets, DIY-ed a pair of AR glasses that can recognize custom gestures using Raspberry Pi.

Record the gestures you want to set, and you can achieve cool operations.

I have a bold idea!

The World of DIY AR Glasses

Let’s start the performance!

Pinch your fingers up and down to complete the brightness adjustment command. (This is your first-person perspective)

Gesture recognition is superimposed and displayed in the lens imaging.

Let’s take a more direct perspective and see the effect through the glasses.

DIY Process

The AR glasses themselves are full of a sense of technology, filling the real world with a cyberpunk flavor.

Why not make it cooler? Just snap your fingers to run commands, isn’t this amazing?

Let’s get started; first, we need to design what parts the device will include.

In addition to the glasses frame, the hardware part includes a lens group, 0.6 mm PETG projection lens, and the accessory part is made of PLA material through 3D printing.

After all, it is a DIY smart device, how can we not invite the versatile mini computer Raspberry Pi to the stage?

As for the software part, the gesture recognition program relies on the Python open-source project MediaPipe.

In addition, Teemu Laurila also wrote two program scripts.

One is an application example that controls brightness by pinching fingers, and the other captures gestures in real-time video to be sent to the computer for processing and superimposed display through smart glasses.

All conditions are ready, so let’s assemble and try it out.

After several adjustments, the parts finally combined into the following device.

To make the program usable on the device, first, a Raspberry Pi is needed as program support.

Then set up memory, drivers, runtime environment, multimedia interfaces, networks, etc., to allow the whole device to run at overclocking.

After preparing the hardware and software environment, debug the application program.

The core function of the application program—the gesture recognition model consists of three frameworks, including the BlazePalm model (for recognizing the overall framework and direction of the hand), the Landmark model (recognizing the 21 three-dimensional hand node coordinates), and the gesture recognition model (classifying the recognized nodes into a series of gestures).

During the training process of the recognition algorithm, the BlazePalm model recognizes the initial position of the palm, optimizing real-time recognition for mobile devices.

Within the range of the palm recognized by BlazePalm, the Landmark model identifies the coordinates of 21 three-dimensional nodes.

The gesture recognition model then identifies the state of each finger based on joint angles, mapping the state to predefined gestures, predicting basic static gestures.

Through the Raspberry Pi Zero W, gesture information is captured. Image information is transmitted to the computer, processed by the gesture recognition AI. Then it is sent to the device to issue corresponding gesture commands and synchronized in the projected image.

Its Past and Present

Wait a minute, with a camera, a mini projector, and a computer processor, and it even has a side projection display. This kind of AR glasses seems familiar.

That’s right, even the gesture recognition code used is open-sourced by Google.

Although it does not have the smartphone-like functions of Google Glass, compared to its voice control and touch functions, Teemu Laurila’s smart glasses opted to use custom gestures to trigger commands, adding more of a black technology flavor.

Additionally, while the Google Glass camera is only used for taking photos and videos, Teemu Laurila’s camera also accepts gesture commands and conveys them. At the same time, the projection also chooses a larger square lens for easier observation.

This device is already the second version of the smart glasses completed by Teemu Laurila, optimized in both appearance and performance.

In terms of material selection, the projection lens thickness was changed from 1mm to 0.6mm; PLA material replaced acrylic; a bolted fixed bracket was added, and glue was discarded.

The most important optimization is that the camera uses a square lens to make the image clearer.

Teemu Laurila shared the two additional codes he wrote on the GitHub platform for interested viewers to replicate.

Reference links: https://www.tomshardware.com/news/raspberry-pi-smart-glasses-recognize-hand-gestureshttps://www.youtube.com/watch?v=60Os5Iqdbswhttps://www.youtube.com/watch?v=Gu4oOYo38rQ

GitHub links: https://github.com/Teneppa/CameraStreamhttps://github.com/Teneppa/HandTrackingBrightnessControl

— The End —

This article is original content from NetEase News • NetEase account feature content incentive program signed account 【Quantum Bit】, unauthorized reproduction is prohibited.

「Quantum Bit·Viewpoint」 Live Registration

Where will the industrialization of artificial intelligence go? PhD from Tsinghua, top tech CEO shares online, discussing business opportunities in artificial intelligence from the limitations and capabilities of AI, scan to register~

Click here👇 to follow me, remember to star it~

One-click three connections “share”, “like” and “see”

Technological advancements are seen every day~