TinyML Gesture Recognition Workshop

Have you ever visited the Harry Potter area in Universal Studios Beijing? Do you remember the magical interactive window display that became an instant hit when it opened?

Many tourists lined up in Diagon Alley with wands in hand to experience the thrill of casting spells!

Flowers bloom! Lights shine!

“Wingardium Leviosa” the Quidditch ball takes off!

Unfortunately, due to the current severe pandemic situation, we can’t travel far to Beijing to experience it with a special wand.

But ideas drive everything; even if we are at home, we can still have such magical experiences! All we need to do is move our fingers and create our own “magic wand” with hidden power. By waving and drawing circles, we can cast spells!

The real Hogwarts School of Witchcraft and Wizardry is about to start class. Muggles, let’s learn together!

In this open night, our familiar teacher Light will lead us into the magical classroom, teaching everyone how to use TinyML to create a “wand” that can recognize gestures.

But before we follow Teacher Light to make the wand, we need some foundational knowledge to help us better understand what is so magical about TinyML? How can it support our gesture recognition?

TinyML stands for “Tiny Machine Learning.” It refers to methods, tools, and technologies that allow engineers to implement machine learning on devices operating below the mW power range.

It includes multiple features such as data privacy, network bandwidth, latency, reliability, and energy efficiency. Therefore, it can support various battery-powered devices and applications that need to be always online.

TinyML can become part of our daily lives in various ways.

Applications of TinyML include:

Keyword spotting

Object detection and classification

Gesture recognition

Audio detection

Machine monitoring

This hands-on learning session is about our gesture recognition application! But how does TinyML understand the trajectory of our movements?

When you wave the sensing device in your hand! Draw specific trajectories as if casting spells!

After a series of recognitions, it can identify your trajectory. If we add some feedback devices, like a small light, we can cast a flickering illumination spell just like a wizard!

With the above knowledge, TinyML won’t be a challenge for us. Let’s get our hands moving with Teacher Light and experience making the “wand”!

Teacher Light will guide us through the following aspects! So we can also master TinyML with simple waving actions!

Effect demonstration

Hardware introduction

Project introduction

Framework introduction

Environment setup

Code explanation

The hardware we will use in this workshop is as follows:

Beetle ESP32-C3 (RISC-V Chip)

9250 Nine-axis sensor module