Welcome to the show “You definitely won’t do it after watching, even if you can, you may not have the materials, and even if you have the materials, you may not succeed, and if you don’t succeed, you have to throw it away”
Developers Luis Oliver, Victor Izquierdo, and Alejandro Gutiérrez used the Jetson NANO development kit to create a system called “Drowsiness, Blind Spot, and Emotion Monitor (DBSE)”
This is an in-car assistance system that alerts the driver when they are drowsy or distracted and informs them of objects in their blind spots. The system’s functions are divided into drowsiness detection, emotion detection, and driving monitoring (using Yolov3). The drowsiness and emotion modules use OpenCV’s Haar Cascades method for facial detection. Once the driver’s face is detected, these modules use convolutional neural networks built into PyTorch and run on the Jetson Nano to detect the driver’s eye state (open/closed) and the driver’s emotions. In the “Driving Monitor” module, the Yolov3 algorithm (trained on 80 classes from the COCO dataset) is used to identify objects in the driver’s blind spots.
Project Background
The Centers for Disease Control and Prevention (CDC) states that 35% of drivers in the United States get less than the recommended minimum of seven hours of sleep each day. This primarily affects attention while performing any task and can have long-term health impacts.
According to a report by the WHO (World Health Organization), falling asleep while driving is one of the leading causes of traffic accidents. Up to 24% of accidents are caused by falling asleep, and according to DMV USA (Department of Motor Vehicles) and NHTSA (National Highway Traffic Safety Administration) data, 20% of accidents are related to drowsiness. This is on par with accidents caused by alcohol, and sometimes even worse.
In addition, the National Highway Traffic Safety Administration (NHTSA) mentions that anger or changes in mental state can lead to more dangerous and aggressive driving, endangering the lives of drivers due to these psychological issues.
Prototype Concept
We will create a system capable of detecting a person’s level of drowsiness, with the goal of notifying the user of their state and whether they can drive.
At the same time, it will measure the driver’s ability to maintain attention or attract attention, and whether they are falling asleep while driving. If it detects this state (they are becoming drowsy), a powerful alarm will sound to awaken the driver.
Additionally, it will also detect small vehicles and motorcycles in the car’s blind spots.
Current Status and Opportunities
Current solutions:
-
Mercedes-Benz Attention Assist monitors changes in steering and other driving habits using the vehicle’s engine control unit and alerts the driver accordingly.
-
Lexus has placed a camera in the dashboard that can track the driver’s face rather than the vehicle’s behavior and alerts the driver when their actions seem to indicate they are in a sleep state.
-
Volvo’s Driver Alert Control System is a lane departure system that monitors and corrects the vehicle’s position on the road, then alerts the driver when any drift between lanes is detected.
-
Saab uses two cameras in the cockpit to monitor the driver’s eye movements and reminds the driver with a text message on the dashboard; if they are still drowsy, they will hear a stern audio message.
As you can see, these are all high-end brands, and not a single plug-and-play system is applicable to every car. This is our opportunity because most cars on the road are not in that price range and do not have these systems.
Materials
Hardware:
-
NVIDIA Jetson Nano. x1 https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-nano/
-
Car power inverter.
https://www.amazon.com/s?k=power+inverter+truper&ref=nb_sb_noss_2
-
ESP32. https://www.adafruit.com/product/3405
-
OLED display.
-
Any Bluetooth speaker or Bluetooth audio car system x1
https://www.amazon.com/s?k=speaker&s=price-asc-rank&page=2&qid=1581202023&ref=sr_pg_2
-
USB TP-Link USB Wifi Adapter TL-WN725N x1
-
UGREEN USB Bluetooth 4.0 Adapter x1.
https://www.amazon.com/dp/B01LX6HISL/ref=cm_sw_em_r_mt_dp_U_iK-BEbFBQ76BW
-
HD camera x1 https://canyon.eu/product/cne-cwc2/
-
32 GB MicroSD card x1
-
5V-4A AC/DC adapter power socket connector.
-
VMA204 x1 https://www.velleman.eu/products/view?id=435512
Software:
-
Pytorch: https://pytorch.org/
-
JetPack 4.3: https://developer.nvidia.com/jetson-nano-sd-card-image-r3231
-
YOLOv3: https://pjreddie.com/darknet/yolo/
-
OpenCV: https://opencv.org/
-
Twilio: https://www.twilio.com/
-
Arduino IDE: https://www.arduino.cc/en/Main/Software
-
Mosquitto MQTT: https://mosquitto.org/
Schematic Diagram
All modules use the MQTT protocol to send visual or audio alerts to the mini-OLED display or speaker based on detected information. For example, the mini OLED display shows the type of object in the blind spot, and LEDs on either side of the display indicate the object’s relative position. Similarly, when the driver is distracted, the speaker will also sound an alarm.
Finished Prototype
Blind spot detection result sample: Bicycle detected on the right side of the vehicle
Installed in the car:
Alarm information:
For a detailed introduction to this project, you can see here: https://github.com/altaga/DBSE-monitor
More Jetson NANO projects
Middle school students use Jetson NX to make an AI vision tester
In the project of recognizing road signs with NANO, this is how we did it…
What’s the point of making a JetBot car? It’s more fun to make one with LEGO…
Making an intelligent TV with Jetson NANO, this operation is a bit interesting
A foreign guy made a rover with Jetson NANO, looks so simple…
What a lonely soul, to make a chess robot with Jetson NANO
Watch how this genius dad made a robot to take care of kids with Jetson NANO
[Jetson Development Project Showcase] Use AI to cleverly maintain social distance
[Jetson Development Project Showcase] Malaria detection using open microscopy and deep learning
[Jetson Development Project Showcase] Adaptive traffic control system
[Jetson Development Project Showcase] Simto – Real: Virtual guidance for robot navigation
[Jetson Development Project Showcase] Creating a reading device for the blind with Jetson NANO
[Jetson Development Project Showcase] Detecting plant diseases with Jetson NANO
Taking the first step towards precision agriculture: Creating an intelligent agricultural robot with Jetson NANO
[Jetson Development Project Showcase] Building an underwater vehicle with Jetson Nano
Don’t ask me how many stars there are, Jetson NANO tells you there are many….
Using Jetson NANO, create a Christmas tree that can save bees
Jetson NANO application case showcase | Creating an intelligent commercial store
Jetson NANO application case showcase | Drone positioning flight in dark scenes
[Jetson Development Project Showcase] Using Jetson NANO and TensorRT to create a smart road pothole detector
[Jetson Development Project Showcase] Creating a smart delivery box prototype with Jetson NANO
[Jetson Development Project Showcase] Dare to play rock-paper-scissors with the NVIDIA JetBot car?
[Jetson Development Project Showcase] Implementing facial recognition attendance with Jetson NANO
