For collaboration needs during the internal testing phase of the product, please contact the editor via WeChat: chailiren27

1 Product Introduction

EasyMv Visual Inspection System is an embedded visual inspection system developed by Beijing Muwei Technology Visual Studio. The purpose of this system is to simplify the visual inspection process, reduce the difficulty of visual inspection, lower the cost of visual inspection, and promote the further popularization of machine vision in industrial production.

The EasyMv visual inspection system consists of visual inspection software and a development board. All algorithms in the visual inspection software are independently developed by the studio and have independent intellectual property rights. It adopts a C++ combined with QT development architecture, which has the advantages of high detection efficiency and low resource consumption, making it convenient for the visual inspection architecture to expand to systems like Windows and Android. The development board uses the mainstream open-source development board Raspberry Pi 4B (4G), which has the advantages of a good development ecosystem and stable operation. This development board supports various modes of IO interfaces and can meet the triggering detection and result output functions for four cameras.

2 Hardware Configuration Description

2.1 Development Board Description

The EasyMv visual inspection system uses Raspberry Pi as the development board, and the development board architecture is shown in Figure 1.

The system power is connected to a 5V3A Type-C interface, and USB devices such as cameras, mice, or keyboards can be directly connected to the USB interface. The monitor is connected to the display via a micro HDMI adapter. The 40-pin header is the GPIO interface, and specific definitions can be seen in 2.2 IO Configuration.

2.2 IO Configuration

To facilitate user use, the EasyMv visual inspection system has preset the functions of each IO port. Users only need to wire according to the requirements to complete the detection trigger and result output functions.

Figure 2 IO Configuration Diagram

As shown in Figure 2, camera 1 trigger, camera 2 trigger, camera 3 trigger, and camera 4 trigger are the trigger input signals for the four cameras. When the system detects the pin changing from low to high, it triggers the system detection. After the detection is completed, it outputs the corresponding OK or NG position of the result value. The pin level changes from low to high, and when the input signal changes from high to low, the system outputs a reset signal.

The total OK and total NG output pins are used to output the combined detection results of multiple cameras, that is, when all cameras detect OK simultaneously, it outputs total OK; otherwise, it outputs total NG.

The input voltage of the IO port is 3.3~5V, the output voltage is 3.3V, and the output current is 16mA.

Note: When connecting to the IO port, please be careful not to exceed the voltage or current to prevent damage to the development board.

2.3 Supported Camera Models

The EasyMv visual inspection system supports USB cameras from Hikvision, Meidivision, and Dushe. Customers can choose the camera model according to their needs.

Each development board can support up to 4 cameras.

If the cameras from the above three manufacturers do not meet the requirements, please contact the studio for camera brand expansion.

3 Software Function Description

3.1 Software Startup

The EasyMv visual inspection system software is embedded in the development board, and the software automatically starts after the system is powered on. The software uses a registration code system (one board, one code). If the development board has been registered, the software automatically enters the last opened project for operation. If the development board has not been registered, the software displays the registration window as shown in Figure 3. Please send the machine code to our studio staff to request the registration code for registration.

Figure 3 System Registration Window

If the software is successfully registered, it enters the software system interface. After the software is opened, it defaults to loading the project that was running when the software was closed. After the project is loaded, it automatically enters the running state (opening cameras, IO cards, etc.), and real-time detection can be performed. The software has an administrator mode and a user mode. The administrator mode can set the window layout and various detection parameters.

Figure 4 Software Running Window

3.2 Interface Introduction

The program is divided into a system toolbar, a window display area, and a parameter setting area. The system toolbar can set the project, system permissions, and window layout. The window display area is the image display window area, displaying the camera image and setting the position of the relevant detection box. The parameter setting area mainly displays the detection status and results of the image. The parameter setting area can complete the display of detection data, hardware parameters, detection box parameters, calculation methods, and system debugging information.

Figure 5 System Interface Partition Diagram

3.3 System Toolbar

The system toolbar contains five functional buttons: project, system, run, layout, and help, as well as the system minimize and close buttons.

3.3.1 Project Button

The project button, as shown in Figure 6, mainly completes the functions of creating, saving, loading a new system project, and exiting the system.

Figure 6 Project Button Function

Create New Project: Create a new project. If there is an existing project, it will automatically clear the current system settings.

Save Project: Save the current project. If the current project is a newly created project, the first time saving requires selecting the project save path and setting the project name. When the project is saved, it can automatically open the current project the next time the software is opened.

Load Project: Load a saved project.

Exit System: Close the system software.

3.3.2 System Button

The system button, as shown in Figure 7, mainly includes permission switching functions and historical data viewing functions.

Figure 7 Project Button Function

User: The user button indicates that the current permission is administrator permission and can switch to user permission. When in user permission, this button displays as administrator, allowing a switch back to administrator permission. The administrator permission can set the detection box position and system parameters, while the user permission cannot change the detection box position and system parameter settings, only having detection and display functions.

Historical Data: You can view the detection data saved during this software session, effective only when the detection data saving is set.

3.3.3 Run Button

Run: Indicates the system is simulating operation and can perform system operation detection without IO triggering.

3.3.4 Layout Button

The layout button, as shown in Figure 8, mainly includes camera layout setting function, activation priority function, and hiding toolbox function.

Figure 8 Layout Button Function

Camera Layout: The camera layout refers to setting the number and layout of the camera windows. Clicking the camera layout toolbox displays the settings page as shown in Figure 9. You can complete the camera layout and quantity settings by setting the number of cameras per row.

Figure 9 Only display one camera in the first row

Figure 10 Only display one camera in the first row, and two cameras in the second row

Activation Priority: To facilitate the detection box parameter settings, setting activation priority allows the current window to be maximized, while other windows are displayed in a list format below, as shown in Figure 11. Right-clicking the mouse to double-click the window list below can switch the active window.

Figure 11 Window Activation Priority

Hiding Toolbox: You can hide the parameter setting bar on the right side, suitable for situations where detection data display is not needed.

Figure 12 Toolbox Hidden

3.3.5 Help Button

The help button allows viewing the system user manual and system version.

Figure 13 Help Button

3.4 Camera Window

Each camera has a separate camera window, which features detection image display, detection box position size setting, camera status display, trigger status display, and processing result display functions.

As shown in the figure, the camera window includes an image display area, a toolbar above, and a status bar below.

Figure 14 Camera Window

3.4.1 Camera Toolbar

The camera toolbar is located at the top of each window, and its functions include camera window size adjustment buttons and detection box addition buttons. As shown, the three icons on the left side of the camera toolbar are for enlarging the image, reducing the image, and restoring the initial size. The three icons on the right side are for adding rectangular detection boxes, adding circular detection boxes, and adding line detection boxes.

The icon for adding a rectangular box adds a detection box at the center of the camera window, which can be adjusted in size and position using the mouse. The same method can be used to add circular and line detection boxes.

3.4.2 Camera Status Bar

The camera status bar is located at the bottom of each window. Three indicator lights indicate the camera connection status, trigger detection status, and system detection result status, respectively.

The camera connection status indicator light is red when the current camera is not connected and green when the camera is connected;

The trigger detection status indicator light is red when the current input signal is low and green when the input signal is high;

The system result status indicator light is red when the current detection result is NG and green when the current camera detection result is OK.

3.5 Parameter Setting Toolbox

The parameter setting toolbox is located in the software parameter setting area and contains five pages: detection data, hardware settings, box parameters, calculation methods, and debugging information.

3.5.1 Detection Data

The detection data page mainly displays the system detection results, divided into total detection results and detection data display. The total detection result refers to the summary of detection results when multiple cameras detect simultaneously. If all cameras detect OK, it shows as OK; otherwise, it shows NG. When the system has no input signal detection, it shows as WT. The detection data is displayed in a list format, showing the detection data name, detection result value, and the error from the standard set value.

Figure 15 Detection Data Page

3.5.2 Hardware Settings

The hardware settings mainly include the settings of camera-related parameters and system trigger time settings.

This page allows you to select the current camera’s corresponding camera name and set the trigger form and attributes of the camera.

You can save the camera image and import the camera window image.

You can set the trigger delay time, which is the time for the system to detect after receiving the trigger signal. Setting it to 0 means the system will detect immediately after receiving the trigger signal.

The output hold time is the reset time for the output signal after the camera completes detection, and setting it to 0 means the output signal resets when the input signal is set to 0.

The test button indicates that the current camera is triggered for detection.

Figure 16 Hardware Settings Page

3.5.3 Box Parameters

The box parameters page mainly sets the parameters of the detection box, including the tracking reference, detection method, and related detection parameters.

Figure 17 Box Parameters Setting Page

As shown in Figure 17, the drop-down box in the upper left corner can select the detection box for the set parameters, or it can be directly selected on the image using the left mouse button.

The checkboxes for X reference and Y reference indicate whether to enable the detection box tracking function. X reference represents horizontal tracking, and the tracking point can be selected in the right drop-down box. After selection, click the set reference button to complete the tracking function settings. When the position of the tracked detection box shifts, the detection box follows the tracked detection box.

The detection box can choose from four detection methods: Blob, Fitting, Caliper, and Template.

1） Blob Detection Method

The Blob detection method refers to binarizing the specified area of the image to obtain the template area, then calculating the position, area, and other feature values of the target area.

Figure 18 Blob Parameter Setting Page

As shown in Figure 18, the extraction area is the area where the gray value is between the low threshold and the high threshold. The burr and notch categories can set the size for removing burrs or filling notches in the area, that is, performing morphological closing or opening operations on the area.

Figure 19 Burr/Notch Category Settings

Selecting Maximum Area: Checking the option to select the maximum area means that when multiple areas are extracted, the maximum area will be selected as the target area.

The calculation methods for Blob detection results include ten methods: upper, lower, left, right, upper left, lower left, upper right, lower right, center, and area, which calculate the upper midpoint, lower midpoint, left midpoint, right midpoint, upper left corner, lower left corner, upper right corner, lower right corner, center of the area, and area of the region.

Figure 20 Blob Calculation Method

Figure 21 Blob Detection Results

As shown in Figure 21, the detection box detection results, where the blue contour indicates the detection area. The orange area is the target area, and the red cross in the middle is the detection result, calculated using the midpoint method on the right.

Note: The detection box cannot be set as a Blob detection method when it is a line.

2） Fitting Detection Method

The fitting detection method refers to detecting multiple target points through the detection box and then fitting them into a line or circle. Therefore, first, set a small detection rectangle on the detection box, that is, the detection element. By setting the number, length, and width of the detection element, you can set the edge area and then extract the detection points based on parameters such as gradient, thickness, direction, and order.

Figure 22 Fitting Parameter Setting Page

The number, width, and length of the elements can change the number and size of the cyan detection rectangles shown in Figure 23.

Figure 23 Detection Element Diagram

Edge Gradient: The gradient value of the detection element’s gray statistics can extract the detection points by calculating the derivative value.

Edge Thickness: The calculated value of the detection element’s gray derivative. When the edge is relatively blurred, this value can be increased appropriately.

Edge Direction: The calculation of detection points can be based on whether it is from black to white or from white to black.

Edge Order: The calculation of detection points can be based on whether it is the first or last point that meets the requirements.

As shown in Figure 24, the yellow line indicates the detected detection points, and the red line indicates the fitted line, which is the collection of detection results.

Figure 24 Fitting Detection Results

Note: The detection box cannot be set as a fitting detection method when it is a rectangle.

3） Caliper Detection Method

The caliper detection method detects single edge points based on a reference line. Using a line as a reference, the element width is the gray statistical width, calculating the edge gradient, and then calculating the target point based on edge direction and order.

Figure 25 Caliper Detection Method Diagram

Edge Gradient: The gradient value of the detection element’s gray statistics can extract the detection points by calculating the derivative value.

Edge Thickness: The calculated value of the detection element’s gray derivative. When the edge is relatively blurred, this value can be increased appropriately.

Edge Direction: The calculation of detection points can be based on whether it is from black to white or from white to black.

Edge Order: The calculation of detection points can be based on whether it is the first or last point that meets the requirements.

As shown in Figure 25, the red cross indicates the caliper detection result.

Figure 25 Caliper Detection Result Diagram

4） Template Detection Method

The template detection method refers to the shape template matching method, which can perform image rotation positioning or matching score and matching position detection based on the template matching result.

Figure 26 Template Detection Parameter Diagram

Starting Angle: The starting angle for template searching;

Ending Angle: The ending angle for template searching; (the starting and ending angles set the range of rotation angles for matching feature images)

Angle Step: The angle step for template matching. A larger step results in shorter detection time but lower matching accuracy. Conversely, a smaller step results in longer detection time but higher matching accuracy.

Edge Gradient: The threshold for extracting the matching feature shape contour.

Learn Template Button: Based on the starting angle, ending angle, angle step, and edge gradient settings, complete the template training.

Minimum Score: The minimum score for matching the template. Features with scores lower than this value will be excluded from matching.

Positioning Image: Whether to perform image rotation positioning based on the matching result.

When performing template positioning, first draw a rectangular detection box around the template feature, then set the starting angle, ending angle, angle step, and edge gradient, and then perform template learning. After template learning, set the matching score and click the parameter setting button.

Note: When modifying the starting angle, ending angle, angle step, or template matching position, you need to redo the template learning. If only the minimum score is modified, you do not need to redo the template learning, only click the parameter setting button.

3.5.4 Calculation Method

The calculation method refers to the integrated analysis of the detection results of the detection box, setting the detection data name, test data, reference data, test method, reference value, upper deviation, and lower deviation scale.

Figure 27 Calculation Method Interface

Name: Test data name, displayed in the detection data page, can be freely edited.

Test Data: The detection data, which can be a single data point or line.

Reference Data: The testing reference of the detection data can be a single data point or line.

Method: The method for detecting data can be set to single value (Value), horizontal distance (HDis), vertical distance (VDis), point-line distance (P2L), line-line distance (L2LD), line-line angle (L2L).

Reference Value: The standard value for the detection result data;

Upper Deviation: The upper tolerance for the detection data relative to the standard data;

Lower Deviation: The lower tolerance for the detection data relative to the standard data;

Increase Column, Modify Column, Clear Column are buttons for editing data. Clicking the button enters the calculation method editing window, where data can be edited according to the corresponding prompts.

Figure 28 Calculation Method Setting Interface

After completing data editing, click the save button to fill the value into the list box of the calculation method page.

After completing the calculation method settings, click the save button on the calculation method page to complete the calculation method settings.

Saving Test Data: You can save the detection data and view historical data through the system data in the system tool menu.

3.5.5 Debug Information

The debug information page, as shown in Figure 29, allows you to display debug information for each process by checking the debug information checkbox below.

Normal debug information is in black font;

Alarm debug information is in yellow font;

Error debug information is in red font.

Figure 29 Debug Information Interface

End

▼Long press the QR code below to recognize and follow for free