Based on OpenCV, this project involves the Raspberry Pi controlling the balance of a small ball, recognizing and locating the ball’s position on the platform, and sending the data to the STM32F103 microcontroller via serial port.

Overall Plan

• Fiberglass board as the platform

• If a servo ball head connecting rod is used, the installation of the servo bracket and the support for the swing arm’s pivot must be designed.

• Servo Drive

• Standard servos: require a controllable angle that is not too large, so they can be considered. Ball head connecting rod drive, with variable torque (servo position adjustable).

• Direct drive is not very realistic.

• Raspberry Pi with OpenCV camera/resistive screen to locate the ball.

Computer Vision Solution

1. Threshold segmentation to extract the platform.

2. Detect the platform’s contour to obtain four corner points.

3. Perform perspective transformation to eliminate the influence of the platform’s angle.

4. Globally search for the minimum value to locate the ball’s position.

5. If the minimum value is large, it is considered that the ball has fallen.

Development Environment

• Set up a Linux OpenCV cross-compilation environment on Windows.

• Raspberry Pi equipped with OpenCV 3.2 and fisheye lens.

• During debugging, use socket + JPEG to send frame sequences to the Windows server application; under black and white 640*480 resolution, it can reach 30 frames.

References

• Real-time video transmission based on Socket and OpenCV (On Windows)

• Real-time video transmission based on Socket and OpenCV (On Linux)

OpenCV_raspberry

Compiling OpenCV for Raspberry Pi, based on VisualGDB project, does not include OpenCV source code and compilation results!!

Simple Usage

Build an OpenCV 3 cross-compilation project based on the official tutorial, all software installed in the default directory.

Raspberry Pi Configuration

1. Install the image file configured with the OpenCV development environment and dependency libraries based on the official tutorial.

Windows Configuration

1. Install and crack VisualGDB 5.2r8 (based on previous experience, version 5.1 cannot recognize certain dependency libraries in the Raspberry Pi system directory).

2. Unzip the packaged OpenCV source code and compiled dynamic link libraries to D:\Program Files\OpenCV_3_2_0_source (this is the OpenCV source directory configured in this project, cannot be changed).

3. Install the Windows toolchain for Raspberry Pi provided on the official website. Download 4.9.2 2016-09-23-raspbian-jessie (Raspberry Pi 1/2/3/Zero) raspberry-gcc-4.9.2-r4.exe (738 MB) and install.

4. Directly open the Visual Studio project file, right-click the project name – VisualGDB properties – Project settings – Deployment machine, set the target Raspberry Pi’s IP and account, password.

5. Right-click the project name – VisualGDB properties – CMake project settings – Sync sysroot – OK, which takes about 1 hour. Alternatively, you can directly copy the sysroot folder to C:\SysGCC\Raspberry\arm-linux-gnueabihf directory.

6. At this point, you can compile. If there is an error, you may need to run ‘make install’ on the Windows side according to step 10 of the official tutorial.

Experience and Resources

Raspberry Pi Configuration:

1. Install the operating system.

• Ubuntu MATE: https://ubuntu-mate.org/raspberry-pi/

• Raspbian: https://www.raspberrypi.org/downloads/

Perform simple configuration on the operating system.

• Set up the Raspberry Pi machine vision programming environment from scratch: http://blog.csdn.net/iracer/article/details/51620051

Install OpenCV-related libraries.

• [Tutorial] Initial configuration after installing OS on Raspberry Pi, taking OpenCV 3.1.0 as an example: http://www.jianshu.com/p/7afe8bfa26c0

• Set up the Raspberry Pi machine vision programming environment from scratch: http://blog.csdn.net/iracer/article/details/51620051

VS Configuration:

1. Install Visual Studio.

2. Install the latest version of VisualGDB.

3. Create a Linux project.

• Cross-compile OpenCV 3 for Raspberry Pi 2.

• Use the OpenCV library on Raspberry Pi.

• Developing a Raspberry PI app with Visual Studio.

• Basic tutorial:

• Using the Windows cross-compilation scheme:

How to call the Raspberry Pi camera using OpenCV:

• Using OpenCV with Raspberry Pi 2 camera.

• [Using Raspberry Pi 2 camera with C++ program in Visual Studio](https://visualgdb.com/tutorials/raspberry/camera/).

About the Windows toolchain for Raspberry Pi:

• Windows toolchain for Raspberry Pi.

Official compatible images provided by VisualGDB:

• Raspberry Pi / Raspberry Pi 2 Jessie Image.

About other Raspberry Pi libraries:

• Using WiringPi library with Raspberry PI cross-compiler.

• Create “Blink LED” project for Raspberry PI.

VS Configuration Issues

After configuring according to the cross-compilation tutorial for OpenCV 3 for Raspberry Pi 2, the compilation fails? The compilation reaches halfway and prompts “lib… needed by … not found” (the final solution fixes the path link issue with the cross-compiler).

• Checked that pkg-config-lite for Windows was downloaded and extracted to the \ Raspberry \ bin directory.

• Checked that gtk2.0 was installed via sudo apt-get install libgtk2.0-dev.

• Checked that /usr/share/pkgconfig and /opt/vc directories were included when synchronizing sysroot.

• Set the environment variables for cmake: PKG_CONFIG_SYSROOT_DIR=C:/SysGCC/Raspberry/arm-linux-gnueabihf/sysroot|PKG_CONFIG_PATH=C:/SysGCC/Raspberry/arm-linux-gnueabihf/sysroot/usr/lib/arm-linux-gnueabihf/pkgconfig;C:/SysGCC/Raspberry/arm-linux-gnueabihf/sysroot/usr/share/pkgconfig.

• Similar issues:

• Compilation reaches halfway and prompts lib… needed by … not found.

• Hi, Looks like your toolchain sysroot may have the incorrect linker configuration file. Please try updating to VisualGDB 5.2 and then resynchronize the sysroot. This should repair the config file automatically.

• Next step is to upgrade to VisualGDB 5.2 and try resynchronizing the Raspberry Pi directory.

• But how to add ldflags in cmake? How to change library_names?

• Official documentation: https://sysprogs.com/w/fixing-rpath-link-issues-with-cross-compilers/.

• LD warning: LIB-XYZ, LIB-ABC needs – CROSS COMP RASP PI”.

• According to CANNOT COMPILE RASPICAM.

How to run the compiled binary executable file?

• Add executable permission with chmod: chmod 777 filename.

• Execute: ./filename parameter list.

Examples

OpenCVDemo

Demo example based on the compiled OpenCV library above – Canny edge detection.

Raspicam

Raspberry Pi C++ camera library compilation project, supports OpenCV.

raspicam-0.1.6

Referenced by Raspicam, raspicam source code.

• After compilation, rename the generated dynamic library libraspicam.so to libraspicam.so.0.1, and place it along with libraspicam_cv.so in the /usr/lib/raspicam folder. Regarding the storage location of dynamic libraries in Linux, there are /lib and /usr/lib; what is the difference between the libraries in these two directories? If there is no difference, why are they stored separately?

• Then update the dynamic library search path according to the Linux method for resolving dynamic link library .so file not found.

• For creating folders and moving files in Linux, refer to the usage instructions for copying, deleting, and moving files in Linux.

RaspberryCameraTest

Test the compiled Raspicam library, does not depend on OpenCV.

OpenCVCameraDemo

Edge detection example based on OpenCV and Raspicam.

For the project address, please click to read the full article.