Playing Deep Learning on Raspberry Pi with TensorFlow

Reprint please specify the source: http://www.codelast.com/

Software and hardware environment of this article:

Raspberry Pi: Model B V1.2, 1GB RAM

OS: Raspbian Jessie

Deep Learning is very popular now, and Raspberry Pi enthusiasts certainly won’t miss out. There are already many Raspberry Pi projects that have jumped on the Deep Learning bandwagon. Purely for fun, I experimented with using TensorFlow on the Raspberry Pi to identify what an object in an image is.

『1』 A Brief Introduction to Deep Learning

The following explanation is from Wikipedia:

Deep Learning is a subfield of machine learning that attempts to model high-level abstractions in data using multiple processing layers composed of complex structures or multiple non-linear transformations.

The applications of Deep Learning are extensive, with the most commonly known applications related to people’s livelihoods being: facial recognition, speech recognition, image search, online translation, etc.

Currently popular deep learning frameworks include TensorFlow (open-sourced by Google), MXNet (supported by Amazon), Theano, etc. Utilizing these frameworks, we only need to do relatively little work to bring deep learning capabilities into our own programs.

Source: http://www.codelast.com/

『3』 Deep Learning on Raspberry Pi

Currently, mainstream deep learning frameworks are not primarily prepared for mobile/embedded platforms—this is due to limitations in computing power. Running local deep learning programs on mobile platforms is often very slow. Therefore, mobile platforms mainly adopt a model of submitting computing requests to the cloud, processing data after cloud computing is completed, and returning results.

As a “quasi-embedded” platform, the Raspberry Pi is the most well-known product in its category, but I believe that in the world of deep learning, the Raspberry Pi has not yet reached the level of “supporting app development” like Android or iOS.

Thus, getting any mainstream deep learning framework to run on the Raspberry Pi will be a time-consuming and labor-intensive task.

Fortunately, TensorFlow is so popular, and there are always talented individuals in the IT industry who have successfully “ported” it to the Raspberry Pi 3 (see here), so using TensorFlow to implement deep learning applications on the Raspberry Pi is a good choice.

Source: http://www.codelast.com/

『3』 Installing TensorFlow on Raspberry Pi

According to the author’s documentation, you can get TensorFlow running on the Raspberry Pi with just a few extremely simple steps. If you’re interested, you can directly check out the tutorial written by the author.

First, I must declare:

• The author used the most popular Linux distribution Raspbian on the Raspberry Pi, while I used Arch Linux ARM, but it doesn’t matter, as my tests showed no issues (at least I didn’t encounter any).

• Due to developing other programs on the Raspberry Pi, I had already pre-installed a number of development libraries/software, such as Protocol Buffers, NumPy, pip, etc., and some of these may be dependencies for TensorFlow, so I did not need to install packages like pip, python-dev as mentioned in the author’s documentation.

Source: http://www.codelast.com/

Then the most critical step is to download a wheel file from GitHub and install it.

A wheel is one of the formats for Python software installation packages; essentially, it is a zip package format with the .whl extension used for installing Python modules.

If you are using Python 2.7:

wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v0.11.0/tensorflow-0.11.0-cp27-none-linux_armv7l.whlsudo pip install tensorflow-0.11.0-cp27-none-linux_armv7l.whl

If you are using Python 3.3+:

wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v0.11.0/tensorflow-0.11.0-py3-none-any.whlsudo pip3 install tensorflow-0.11.0-py3-none-any.whl

The download process for the whl installation package may take a while. I used Python 3.5.2, and below is my command line output:

Processing ./tensorflow-0.10.0-py3-none-any.whl

Collecting protobuf==3.0.0b2 (from tensorflow==0.10.0)

Downloading protobuf-3.0.0b2-py2.py3-none-any.whl (326kB)

100% |████████████████████████████████| 327kB 11kB/s

Collecting wheel>=0.26 (from tensorflow==0.10.0)

Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)

100% |████████████████████████████████| 71kB 18kB/s

Requirement already satisfied (use –upgrade to upgrade): numpy>=1.8.2 in /usr/lib/python3.5/site-packages (from tensorflow==0.10.0)

Requirement already satisfied (use –upgrade to upgrade): six>=1.10.0 in /usr/lib/python3.5/site-packages (from tensorflow==0.10.0)

Requirement already satisfied (use –upgrade to upgrade): setuptools in /usr/lib/python3.5/site-packages (from protobuf==3.0.0b2->tensorflow==0.10.0)

Installing collected packages: protobuf, wheel, tensorflow

Successfully installed protobuf-3.0.0b2 tensorflow-0.10.0 wheel-0.29.0

You are using pip version 8.1.2, however version 9.0.1 is available.

You should consider upgrading via the ‘pip install –upgrade pip’ command.

Source: http://www.codelast.com/

So what’s next? Then you’re all set! It’s that simple!

If the above steps fail, you can only choose to compile TensorFlow from source, which is quite a troublesome task, and I can only wish you good luck. However, the good news is that the author has already helped everyone avoid many pitfalls, and the tutorial is here.

『4』 Using TensorFlow on Raspberry Pi

To identify what an object in an image is, we first need to train an image classification model, which is very resource-intensive and not wise to do on the Raspberry Pi. We can directly use the Inception-v3 model that Google has already trained.

Inception-v3 is trained for the ImageNet Large Visual Recognition Challenge using the data from 2012. This is a standard task in computer vision, where models try to classify entire images into 1000 classes, like “Zebra”, “Dalmatian”, and “Dishwasher”.

First, download the Inception-V3 model to any directory and unzip it:

mkdir ~/tensorflow-related/modelcd ~/tensorflow-related/modelwget http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz tar xf inception-2015-12-05.tgz

Unzipping yields a bunch of files:

classify_image_graph_def.pb

cropped_panda.jpg

imagenet_2012_challenge_label_map_proto.pbtxt

imagenet_synset_to_human_label_map.txt

LICENSE

Then you can start performing image recognition! Here I found a popular image of a Mobike online:

Please click here to enter a description of the image

Source: http://www.codelast.com/

Use TensorFlow to recognize it:

cd /usr/lib/python3.5/site-packages/tensorflow/models/image/imagenetpython3.5 classify_image.py --model_dir /root/tensorflow-related/model --image_file /root/tensorflow-related/test-images/mobike.jpg

Here, the path /usr/lib/python3.5/site-packages/tensorflow/models/image/imagenet is where TensorFlow’s Python image classification program classify_image.py is located; it may vary by OS.

The –model_dir parameter is the path to the model files we unzipped earlier, and –image_file is the path to the image to be recognized.

The output is as follows:

/usr/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py:1750: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future

result_shape.insert(dim, 1)

W tensorflow/core/framework/op_def_util.cc:332] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().

bicycle-built-for-two, tandem bicycle, tandem (score = 0.33731)

tricycle, trike, velocipede (score = 0.16082)

unicycle, monocycle (score = 0.12926)

mountain bike, all-terrain bike, off-roader (score = 0.10689)

parking meter (score = 0.01563)

[root@alarmpi imagenet]# python3.5 classify_image.py –model_dir /root/tensorflow-related/model –image_file /root/tensorflow-related/test-images/mobike.jpg

/usr/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py:1750: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future

result_shape.insert(dim, 1)

W tensorflow/core/framework/op_def_util.cc:332] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().

mountain bike, all-terrain bike, off-roader (score = 0.56671)

tricycle, trike, velocipede (score = 0.12035)

bicycle-built-for-two, tandem bicycle, tandem (score = 0.08768)

lawn mower, mower (score = 0.00651)

alp (score = 0.00387)

It can be seen that TensorFlow believes the image is a mountain bike/all-terrain bike/off-roader with a probability of 0.56671; the recognition result is quite good.

Source: http://www.codelast.com/

『5』 Speed Issues

The image recognition test above took over 50 seconds! Such a slow speed has basically no practical value in real applications.

However, this time can be significantly reduced.

• Every time we run the program, we reload the model, and the model file is large, which wastes a lot of time. Therefore, the program should be made to run as a resident memory program, loading the model only once, which can save a lot of time for each image recognition.

• It is necessary to “preheat” the model. According to the performance test results of the author and others, reasonable preheating can significantly reduce computation time.

• Using TensorFlow’s C++ interface to implement the program should be faster than the Python version, so it can be tried.

• Currently, TensorFlow cannot utilize the Raspberry Pi’s GPU for computation (see here), but we hope that one day this wish will be realized.

So at least the terrifying figure of over 50 seconds can be avoided.

Source: http://www.codelast.com/

『6』 What is the Use of Running TensorFlow on Raspberry Pi

One potentially meaningful application that I randomly thought of is: a child learning assistance system. Mount a camera on the Raspberry Pi, and when a child holds something in front of the camera, the camera takes a picture, recognizes what it is, and reads out the most probable English word.

Of course, there are many engineering problems that need to be solved here.

I believe that as the computing power of the next generation of Raspberry Pi continues to strengthen, and with each release of TensorFlow improving performance, the practicality of running TensorFlow applications on the Raspberry Pi will be very good in the near future.