Big Data Digest authorized reprint from Academic Headlines

Compiled by: Wu Lingzhu

Can facial recognition recognize twins?

This is a question that many people have probably thought about, but few have put into practice to explore.

Coincidentally, a technician with a pair of twin sons became interested in this question, so he built a facial recognition system using Raspberry Pi, trying to use AI to recognize his sons!

The result is obvious, as the two sons look too alike, the AI recognition is not as accurate as his own recognition.

He documented the entire experimental process in a blog, and the following is the full text of the blog.

By 2021, people generally believe that through learning algorithms and artificial intelligence research, machines outperform humans in many aspects.

For a long time, machines have dominated the manufacturing and supply chain fields, and have even begun to tackle more complex tasks, such as autonomous driving and even writing high-difficulty comprehensive articles. However, has the development of machines in facial recognition really surpassed the human brain?

Thirteen years ago, when my identical twin sons were born, my wife and I could easily tell which one was Nicolas and which one was Alexander.

After spending 8 weeks in the Neonatal Intensive Care Unit (NICU) at California Pacific Medical Center in San Francisco, we learned how to match their individual behaviors with their names. We learned early on what most parents of identical twins would master: who a person is depends not only on their appearance but also on their walking, speaking, and various behavioral differences in interacting with the world. Processing this broad dataset allows humans to instantly recognize people we know, including identical twins.

AI-Powered Raspberry Pi Face Recognition System for Identifying Twins

Image | Alexander and Nicolas

With widespread applications from identifying criminals to helping us shop, facial recognition has been referred to in the news as a “Matrix-like silver bullet”. I’ve been wondering, what can machines do when the dataset presents only very slight differences? For example, recognizing identical twins.

Diving into the differences

As my sons grow day by day, we discuss the unique nature of their genetic makeup in detail — essentially, they are cloned from the same fertilized egg (Note: there is a very high frequency of “cloning” factor because they are avid fans of the clone army in “Star Wars”).

To them, they are completely identical people, created from the same blueprint.Of course, there are differences between them, and these differences cause me to occasionally confuse them, and I suspect machines would also struggle to tell them apart. The same DNA is one thing, but the intrinsic expression of genes and environmental influences play an important role in human variation and development over time.

For example, I recently bought a smartphone with a fingerprint scanner and asked my sons if they wanted to do a little experiment: one would enroll their fingerprint, and then the other would try to unlock it with their own finger, which clearly would not work. It turns out that a variety of environmental factors (of which only one is genetic) drive the formation of “fingerprints” in utero, so twins typically have different fingerprints.

To help answer the question of “who is better at recognizing identical twins, humans or machines”, I decided to design an experiment slightly more complex than a fingerprint reader and invited my “little scientists” (my sons) to assist.

Building the experiment

For this project, I wanted to build a computer system as cheaply as possible that could run all the software needed to execute and train machine learning and facial recognition models.I chose Raspberry Pi because it is highly cost-effective, has peripheral support, and can run standard Linux distributions and all the tools necessary for compiling software.

I already had some Raspberry Pi units from other projects, so I tested three to see which one was best for this application. The Pi Zero worked fine, but it took 48 hours to analyze the video feed frames and compile all the necessary software (yes, compiling that software took 2 days).

For anyone wanting to run a similar environment, I highly recommend the Pi 4b (8GB model) as it has a faster processor, multiple cores, and expanded memory. The detailed specifications and form factors of the three Pis we tested are as follows. The rest of the setup assumes the use of the Pi 4b (8GB).

Image | Pi Zero, Pi 3b+, and Pi 4b – Scaled Pencil

This project also requires a webcam. I already had a Logitech USB webcam that worked well without needing to install drivers. Any modern Logitech or similar webcam can be used.

Setup: Step 1

The first step in setting up the Pi is to flash Raspberry Pi OS onto the micro SD card and do some basic configuration. To do this, download and install the Raspberry Pi Imager on your workstation. Once installed, insert the micro SD card (usually using an adapter), launch the imager, select the operating system and SD card, and then choose “Write”.

For the OS (operating system), please select Raspberry PI OS (32-bit), which is a port of Debian Linux that contains everything needed to set up a machine learning environment. After the SD card is written and verified, do not remove it; before reinserting it into the Pi, three additional configurations are needed: set up WiFi, SSH, and video output.

WiFi: When the Pi starts, it needs to connect to the local network. Create a file named wpa_supplicant.conf and place it at the top level of the SD card. Make sure the file is in plain text format and ends with .conf. Before saving the file, add the two-letter country code for your location along with the SSID and PSK for your WiFi network (keep them in double quotes).

SSH: Since we are setting up the Pi to run headless (without a monitor), we need a way to connect to it for management and configuration — using SSH. To enable the Linux SSH daemon and automate it, place a file named ssh at the top level of the SD card that contains no content and has no file extension.
Video: We will start the Pi without a display connected, but we still want to enable video while remotely controlling it, so we must force the HDMI to be active even if it is not detected. Edit the config.txt file at the top level of the SD card and uncomment the line hdmi_force_hotplug = 1.

Now eject the micro SD card, insert it into the Pi, connect it to power, and turn it on. After about a minute, the Pi should have fully booted and connected to your WiFi network. Check your WiFi router for the Pi’s IP address. Now, verify you can connect to the Pi by opening a terminal (on Mac) or using Putty (on Windows) and SSH with the IP address as username “Pi” (default password is “raspberry”).

Setup: Step 2

The second step of the installation process is to install and compile all necessary software. To perform facial recognition tasks, I decided to use OpenCV (Open Source Computer Vision Library) and Dlib. OpenCV and Dlib have many optimized algorithms, including facial recognition, and are widely respected and supported in industry and academia, with interfaces available for the Python programming language to drive model training and recognition programs. There are many examples of setting up OpenCV and Dlib on Raspberry Pi on the internet, but I couldn’t find anything concise and comprehensive, so I created a simple wrapper to install all the necessary software and compile the OpenCV and Dlib source code. To download and execute this wrapper, please do the following:

Connect to the Pi via SSH as the pi user and ensure you are in the pi user’s home directory: cd~
Clone a copy of our Raspberry Pi facial recognition repository on GitHub: git clone https://github.com/tomalessi/pi-facial-recognition.git
Execute the installation script (takes about 1 hour): pi-facial-recognition/pi-setup.sh

Setup: Step 3

The Pi is now ready to take pictures and train the facial recognition model. To do this, we need to view the window environment on the Pi so that we can connect to it via VNC Viewer. You need to enable the VNC server on the Pi and download a client on your workstation. Please follow these steps:

Pi Setup: SSH into the Pi, execute the command sudo raspi-config, scroll down to Interface Options → VNC and enable it. After enabling this feature, you will need to reboot.
Workstation Setup: Download and install VNC Viewer from RealVNC for your platform (free download). Once installed, set up a new connection from VNC Viewer to the Pi using its IP address and ensure you can log in. Once logged in successfully, you will see the background of the Raspberry Pi OS.

Performing facial recognition

The facial recognition process involves three steps: taking pictures, training the model, and analyzing real-time video. These three tasks can be executed using the face-req.py Python program that has been popped from the GitHub repository above (Note: this is the first revision and does not have much error handling – if you want us to improve it, please submit a PR). The steps for taking pictures, training the model, and analyzing video are as follows. All three assume you have logged into the Pi via VNC and opened a terminal window.

Take a picture by executing the following command and pressing the space bar while watching the live video: python pi-facial-recognition/face-req.py -c photo -n Tom Continue taking pictures for others by replacing the -n parameter with another parameter. Press ctrl + c in the terminal window to exit.
Train the model: python pi-facial-recognition/face-req.py -c train. It takes about 10 seconds to analyze each photo and serialize the results for video analysis.
Analyze video and apply the facial recognition model: python pi-facial-recognition/face-req.py -c analyze. After about ten seconds, live video should start. All recognized faces will be tagged with the person’s name, otherwise, the face will be tagged as unrecognized. Press “q” to exit.
To clean the environment and start over, please execute: python pi-facial-recognition/face-req.py -c clean.

Experimental results

After fully setting up the environment, I took 3 pictures of myself, 3 pictures of Nicolas, and 3 pictures of Alexander, and then used these photos to train the facial recognition machine learning system (one front view, one left, and one right). On the day of performing these tests, no adjustments were made to any photos other than the current ones. After training the system, we took turns standing in front of the camera to see if the machine could recognize us.

Note: We tried another experiment by taking 10 pictures from different angles, but this did not change the results, possibly because we used a Histogram of Oriented Gradients (HOG) based facial recognition method, which only works for frontal photos.

Results: Test 1

Test 1 is myself and Alexander, the machine recognized correctly.

Results: Test 2

Test 2 is me and Nicolas.The machine recognized correctly again. I began to think my initial guess was wrong — perhaps the machine would win.

Results: Test 3

Test 3 is the “main event” — all three of us together. Then the machine failed! Clearly, there are not two Nicolas.We found that if the kids move around, the recognition toggles back and forth, but the machine rarely recognizes twins simultaneously. Unscientifically, about 5% of the time, it could correctly recognize both twins and myself at the same time.

Results: Test 4

I wanted to see how the machine learning model would recognize photos of the twins at age 3 — the photo was presented in front of the camera. I must give credit to the machine; it did correctly recognize Nicolas in the photo (the first time), although it did not recognize Alexander. I slightly adjusted the orientation and distance of the photo in front of the lens, and then it incorrectly recognized Alexander and failed to recognize Nicolas. However, this is already a good result, as the children’s faces have undergone many changes from ages 3 to 10.

Alexander (left), Nicolas (right)

Results: Test 5

I tried a newer color photo (the kids might be 4 years old), and the machine failed again. It incorrectly identified two Nicolas.

For the final test, I wanted to see what the age limit for these photos should be before the machine could not recognize them at all. I used a photo of the twins around one year old. The machine correctly recognized me but failed to recognize the twins.

Note: Even a static photo at age 12, my wife and I can easily tell the twins apart immediately.

Experimental conclusions

The first conclusion here is obvious. A person can build a fully functional computer system capable of running modern facial recognition and machine learning algorithms for $75 (if you don’t mind a bit slow, some can be built for as little as $10). Not to mention the more complex task of recognizing twins, this machine had a 100% success rate in recognizing non-twins in live video.

The system achieved a 100% recognition success rate for every non-twin trained (we did other non-twin tests not shown here). At the same time, for personal privacy reasons, this does give me pause. This technology is powerful, very easy to use, and is likely to be deployed in ways we cannot foresee in our daily lives — we need to ensure that this technology is not misused.

The second conclusion is that our facial features undergo significant changes during the first 13 years of our lives, making facial recognition difficult for young people.The machine’s ability to recognize twins from old photos, even when they were “toddling”, impressed me. Honestly, that’s amazing! Besides my wife and me, no one can look at our children’s old photos and determine who is who.

Finally, and most importantly, I observed that machines struggle to recognize identical twins, even twins like my children who have obvious differences. The system indeed struggled during recognition, especially when a pair of twins or one is moving within the video frame — misidentifications occurred frequently. From these simple tests, we conclude that for now, humans are much better than machines at facial recognition. This is not surprising, as we have been evolving and refining our senses for 6 million years. I cannot tell you why I can easily distinguish between my children; I just know it is easy. Machines may eventually catch up with us, but not yet.

Next steps

My sons want to build in some additional features on the Pi to control locks on doors or greet them when they enter their bedroom, but based on our results, we first need to enhance the Pi’s ability to distinguish between them (to prevent one from accessing the other’s treasures). To improve the system’s accuracy, I want to try some different facial recognition techniques, such as using convolutional neural networks (CNN) supported by Dlib (although the Raspberry Pi is limited by its computational power) or having the Pi call public services for facial recognition in GCP, AWS, or Azure. Academic research on facial recognition for twins is also increasing, so technology will certainly continue to improve to address some of the existing flaws we pointed out here — my sons and I will definitely keep an eye on this field.

People who click “like” have become more attractive!

AI-Powered Raspberry Pi Face Recognition System for Identifying Twins

By 2021, people generally believe that through learning algorithms and artificial intelligence research, machines outperform humans in many aspects.

Understanding the Relationship Between Micro:bit and Arduino

Getting Started with the Arduino ESP32 Core Library

Implementing Neural Networks on a 10-Cent RISC-V MCU Without a Multiplier

Getting Started with Arduino: Light Up Your First LED from Scratch

How Highly Integrated Embedded Processors Drive the Development of Industrial Robots

Leave a Comment Cancel reply

Menu

<img decoding="async" alt="AI-Powered Raspberry Pi Face Recognition System for Identifying Twins" src="https://ss.boardor.com/wp-content/uploads/2024/12/e26ded604022d6a925f86c402980d5fb5bd2f3e7e31be2eecc668ed82732b704.jpg" data-eio="l" />

By 2021, people generally believe that through learning algorithms and artificial intelligence research, machines outperform humans in many aspects.

Menu