Biometric recognition includes face recognition, voice recognition, fingerprint recognition, retinal or iris recognition. Among these, face recognition is currently the most widely used biometric technology in identity authentication or identity lookup applications.

Face recognition specifically refers to the computer technology that uses analysis and comparison of facial visual feature information for identity verification. Broadly speaking, face recognition actually includes a series of related technologies for building a face recognition system, including face image acquisition, face localization, face recognition preprocessing, identity confirmation, and identity lookup; while narrowly speaking, face recognition specifically refers to the technology or system that verifies or looks up identity through facial features.

Traditional face recognition technology is mainly based on visible light images, which people are more familiar with. However, this method has very obvious disadvantages due to significant lighting limitations. With technological advancements, some new solutions have emerged, such as 3D image face recognition and thermal imaging face recognition.

In recent years, several excellent open-source face recognition projects have emerged in the open-source field. These projects are open-source and free, and they can meet the usage requirements of certain application scenarios. This article focuses on introducing these open-source projects.

01

OpenFace

Github：https://github.com/TadasBaltrusaitis/OpenFace

OpenFace is a tool for computer vision and machine learning that can perform various face recognition detections, including feature point detection, head pose recognition, facial action recognition, and gaze recognition. Additionally, it can achieve real-time face recognition functionality from a webcam without any specialized hardware.

OpenFace is implemented based on Python and Torch neural network algorithms, and its theory comes from facenet.

Its features include:

(1) Facial landmark detection

(2) Facial landmark and head pose tracking

(3) Facial action unit recognition

(4) Gaze tracking

(5) Facial feature extraction

02

face_recognition

GitHub: https://github.com/ageitgey/face_recognition

face_recognition is a powerful, simple, and easy-to-use open-source face recognition project, equipped with complete development documentation and application cases, and compatible with Raspberry Pi systems. This project is a very straightforward face recognition library that can extract, recognize, and manipulate faces using Python and command-line tools. The face recognition in this project is based on a deep learning model from the industry-leading C++ open-source library dlib, tested with the Labeled Faces in the Wild dataset, achieving an accuracy of up to 99.38%. However, the recognition accuracy for children and Asian faces still needs improvement.

Among them, Labeled Faces in the Wild is a face dataset created by the University of Massachusetts Amherst, containing over 13,000 facial images collected from the internet.

This project also provides an easy-to-use face_recognition command-line tool that can process images in an entire folder.

Its main features include:

(1) Locate all faces in an image

import face_recognition
image = face_recognition.load_image_file("your_file.jpg")
face_locations = face_recognition.face_locations(image)

(2) Recognize facial key points, including eyes, nose, mouth, and chin.

import face_recognition
image = face_recognition.load_image_file("your_file.jpg")
face_landmarks_list = face_recognition.face_landmarks(image)

Identifying facial key points is useful in many fields, such as the automatic makeup case of this project (digital make-up): https://github.com/ageitgey/face_recognition/blob/master/examples/digital_makeup.py.

(3) Identify who is in the image

import face_recognition
known_image = face_recognition.load_image_file("biden.jpg")
unknown_image = face_recognition.load_image_file("unknown.jpg")
biden_encoding = face_recognition.face_encodings(known_image)[0]
unknown_encoding = face_recognition.face_encodings(unknown_image)[0]
results = face_recognition.compare_faces([biden_encoding], unknown_encoding)

(4) Real-time face detection

It can be used with other Python libraries (such as opencv) to achieve real-time face detection:

03

InsightFace

GitHub: https://github.com/deepinsight/insightface

InsightFace is an open-source 2D and 3D deep face recognition analysis tool based on PyTorch and MXNet. Compared to MXNet, the implementation based on OneFlow performs better in terms of performance, with OneFlow being 2.82 times faster in data parallelism; 2.45 times faster in model parallelism; and 1.38 times faster in mixed parallel + Partial fc. The code implemented based on OneFlow has been merged into the insightface project repository, which contains dataset creation tutorials, training and validation scripts, pre-trained models, and conversion tools for MXNet models. InsightFace efficiently implements a rich set of state-of-the-art face recognition, face detection, and face alignment algorithms, optimized for training and deployment, achieving high accuracy—99.86% on the LFW dataset.

ArcFace, SubCenter-ArcFace, RetinaFace, etc. are all projects supported by InsightFace.

04

facenet

Github：https://github.com/davidsandberg/facenet

FaceNet is a face recognition system based on deep learning proposed by Google in 2015.FaceNet is a multi-purpose recognition system that can be used for face verification (whether they are the same person), recognition (who this person is), and clustering (finding similar people).When tested using the standard “face database”, FaceNet’s recognition accuracy can reach nearly 100%, maintaining an 86% recognition accuracy even with a massive database of 250 million facial images.The two pre-trained models currently provided achieve 99.05% and 99.65% on LFW.

The basic principle of FaceNet is to directly train a deep convolutional neural network to map facial images into a 128-dimensional Euclidean space, where the distance between different facial image feature vectors is related to the similarity of the images: the smaller the “Euclidean distance” between two facial image feature vectors, the greater the likelihood that the two images are of the same person.

The inference process using FaceNet generally involves the following steps:

• Data preprocessing: Images in conventional face recognition datasets often include the entire head and some environmental background, and faces are often tilted (head tilted, profile). Before inputting such images into the model, it is necessary to crop the “real face” part of the image, remove irrelevant background information, and align the cropped facial images before they can be used for training.

• Load the dataset according to specific rules: The FaceNet system uses a novel data import method: a triplet consisting of two matching images (two images of the same person) and one non-matching image (one image of another person) is loaded as a group.

• Use a large backbone network to extract facial feature information: Select an appropriate deep convolutional neural network, modify the output layer, and facial feature extraction can be performed. Common deep convolutional neural networks include resnet series, Inception series, etc.

• Normalize facial feature information using L2 norm: The output of the deep convolutional neural network is a feature vector in a 128-dimensional hyperspace, which needs to be normalized to the 128-dimensional hypersphere using L2 regularization. Filter out valid features.

• Loss calculation and gradient update: Calculate the triplet loss function, making the model quickly optimize towards the target direction.

05

deepface

Github：https://github.com/serengil/deepface

deepface is a lightweight face recognition and facial attribute analysis (age, gender, emotion, and race) framework in Python. You can apply facial analysis with just a few lines of code. It bridges the gap between software engineering and machine learning research, making it perfect for those who want to drive but don’t know how to build a car.

Deepface is a hybrid face recognition package. It currently wraps many state-of-the-art face recognition models: VGG-Face, Google FaceNet, OpenFace, Facebook DeepFace, DeepID, ArcFace, Dlib, and SFace. The default configuration uses the VGG-Face model.

According to experimental tests, the models that perform better include: FaceNet, VGG-Face, ArcFace, and Dlib. You can find the scores of these models in the datasets declared by their creators, Labeled Faces in the Wild and YouTube Faces in the Wild.

deepface mainly provides the following face recognition algorithms, corresponding to specific functions:

(1) Face verification

(2) Face recognition

dfs = DeepFace.find(img_path = "img1.jpg", db_path = "C:/workspace/my_db")

(3) Facial attribute analysis

DeepFace.stream(db_path = "C:/User/Sefik/Desktop/database")

(4) Face detection

(5) Facial feature extraction

(6) Real-time facial analysis

06

CompreFace

Github：https://github.com/exadel-inc/CompreFace

Exadel CompreFace is a leading free open-source face recognition system. The face recognition algorithm can be easily integrated into any system without prior machine learning skills.

The system provides REST APIs for face recognition, face verification, face detection, landmark detection, mask detection, head pose detection, age and gender recognition; it supports running models on CPU and GPU; The project also includes a role management system to control who has access to the facial recognition service. Additionally, CompreFace provides docker-compose configurations for quickly deploying facial services in Docker mode.

CompreFace supports various face recognition services, including:

(1) Face detection

There are many use cases for face detection, such as: collecting statistics on the popularity of stores among different genders; collecting statistics on which age groups your events are popular; obtaining landmark information to understand customer gaze; collecting statistics on how many customers are in the store; confirming whether all customers are wearing masks correctly.

(2) Face recognition

(3) Face verification

(4) Feature detection

(5) Age recognition

(6) Gender recognition

(7) Mask detection

(8) Head pose detection

(9) User, role, and access management