IoT Platform: Supporting Millions of Connections on a Single Machine

Introduction to ThingLinks Platform

ThingLinks adopts the Spring Cloud microservices architecture, a high-performance, high-throughput, and highly scalable IoT platform! A single machine can support millions of connections while supporting various protocol interactions and customizable extension features, as well as plugin development; it supports connecting massive devices and collecting device data to the cloud; it provides northbound output functions and system APIs, allowing the server to issue commands to the device side via API calls for remote control.

System Technical Architecture

Technology Stack

1. Adopts a front-end and back-end separation model, with the front-end framework VUE.

2. The back-end uses Spring Boot, Spring Cloud & Alibaba.

3. MqttBroker (supports clustered deployment) based on Netty, Reactor3, Reactor-netty.

4. The registry and configuration center use Nacos, and authentication uses Redis.

5. Traffic control framework uses Sentinel, and distributed transaction uses Seata.

6. The time series database uses TDengine, an open-source and efficient big data platform for IoT, processing massive IoT data writes and load queries.

Basic Infrastructure Diagram

Core Features

Device Connection: Supports unified product model management, multiple devices, multiple manufacturers, unified device connection management, and multi-protocol adaptation (MQTT, WebSocket, TCP, UDP, CoAP, HTTP, etc.).

Rule Engine: Device alarms, message notifications, data forwarding.

Visualization: Visual display of device geographical locations, visual large screens.

Time Series Database: TDengine time series database super table design concept: one table for each device, one super table for each type of device.

System Modules

com.mqttsnet.thinglinks
├── thinglinks-ui              // Front-end framework [19000]
├── thinglinks-gateway         // Gateway module [19100], [sentinel:19101]
├── thinglinks-auth            // Authentication center [19200]
├── thinglinks-api             // API module
│       └── thinglinks-api-system                          // System API
├── thinglinks-common          // Common module
│       └── thinglinks-common-core                         // Core module
│       └── thinglinks-common-datascope                    // Permission scope
│       └── thinglinks-common-datasource                   // Multi-data source
│       └── thinglinks-common-job                          // Distributed scheduled tasks
│       └── thinglinks-common-kafka                        // Kafka message service
│       └── thinglinks-common-log                          // Log recording
│       └── thinglinks-common-redis                        // Cache service
│       └── thinglinks-common-rocketmq                     // RocketMQ message service
│       └── thinglinks-common-security                     // Security module
│       └── thinglinks-common-swagger                      // System API
├── thinglinks-modules         // Business modules
│       └── thinglinks-modules-file                        // File service [19300]
│       └── thinglinks-modules-gen                         // Code generation [19301]
│       └── thinglinks-modules-job                         // Scheduled tasks [19302]
│       └── thinglinks-modules-system                      // System module [19303]
│       └── thinglinks-modules-tdengine                    // TDengine service [19304]
│       └── thinglinks-modules-link                        // Link service [19305]
│       └── thinglinks-modules-broker                      // Broker service [19306], [MQTT-TCP:11883], [MQTT-SSL:18443], [MQTT-WS:18999]
│       └── thinglinks-modules-protocolAnalysis            // Protocol analysis service [19307], [TCP-IP:18901], [UDP-IP:18902]
        └── thinglinks-modules-rule                        // Rule module [19308]
├── thinglinks-registry         // Registry micro-application
├── thinglinks-visual          // Graphical management module
│       └── thinglinks-visual-monitor                      // Monitoring center [19400]
│       └── thinglinks-visual-collection                   // Server monitoring collection service [19401]
├── pom.xml                // Common dependencies

Core Function List

System Management: User management, role management, menu management, department management, position management, dictionary management, parameter settings, notifications, log management

System Monitoring: Online users, scheduled tasks, Sentinel console, Nacos console, Admin console, task scheduling management

System Tools: Form building, code generation, system APIs

Device Integration: Device management (supports MQTT protocol, WebSocket protocol, TCP-IP protocol device access), sub-device management, product management, protocol management, rule engine (device linkage)

Device Debugging: Real-time logs, command issuing

Rule Engine Message Forwarding: Supports KAFKA nodes, HTTP nodes, PREDICATE nodes, ROCKET_MQ nodes, RABBIT_MQ nodes, MYSQL nodes, MQTT nodes, TOPIC nodes, LOG nodes

Device Management List

Add Device Information

Device Details

Sub-device Management

Device Shadow

Product Management

Product Model Information

CAS Rule Management

Compilation and Packaging

Confirm that the production environment configuration file is correct (very important)

Maven Compilation

Create Directories

mkdir -p /home/thinglinks
mkdir -p /home/thinglinks/thinglinks-auth
mkdir -p /home/thinglinks/thinglinks-gateway
mkdir -p /home/thinglinks/thinglinks-modules-system
mkdir -p /home/thinglinks/thinglinks-modules-gen
mkdir -p /home/thinglinks/thinglinks-modules-job
mkdir -p /home/thinglinks/thinglinks-modules-file
mkdir -p /home/thinglinks/thinglinks-modules-tdengine
mkdir -p /home/thinglinks/thinglinks-modules-link
mkdir -p /home/thinglinks/thinglinks-modules-broker
mkdir -p /home/thinglinks/thinglinks-modules-protocolAnalysis
mkdir -p /home/thinglinks/thinglinks-modules-rule
mkdir -p /home/thinglinks/thinglinks-visual-monitor

Upload JAR Package

Build Image

# Example (using thinglinks-auth)
# Change directory
cd  /home/thinglinks/thinglinks-auth
# Build (note the command has a dot at the end)
docker build --build-arg JAR_FILE=thinglinks-auth.jar -t thinglinks-auth:latest .

View Images

docker images

Demo Address: http://thinglinks.mqttsnet.com

Username and Password: admin/adminthinglinks

Source Code Address: http://thinglinks.mqttsnet.com