Remote Operation and Maintenance Deployment for Embedded Devices

In on-site operation and maintenance, you may be troubled by the following issues:

  • • Devices are scattered in weak network/cellular/private network environments, unable to open public IPs, and temporary debugging requires sending personnel on-site.
  • • VPN/port forwarding configurations are complex and unstable, making it difficult to implement fine-grained permissions and operation logging.
  • • Device resources are limited (flash memory/RAM/CPU), unable to support heavyweight remote control components.rtty is a lightweight solution designed for the above scenarios.

What is rtty

  • • Architecture：<span>rtty (C client)</span> ↔ <span>WebSocket</span> ↔ <span>rttys (Go server + Web frontend)</span>
  • • Purpose：Access the system terminal of on-site embedded devices remotely through a browser, without relying on public IPs and VPNs; devices are identified by a unique device ID.
  • • Applicable：OpenWrt/Buildroot/Yocto/any embedded Linux, low storage/memory usage, stable operation.

Illustration:

┌──────────────┐        wss/ws         ┌──────────────┐
│  rtty client │  ───────────────────▶ │    rttys     │ ──▶ Browser UI
│  (device)    │ ◀───────────────────  │ (server)     │ ◀──┘
└──────────────┘    keepalive+auth      └──────────────┘

Server rttys Deployment

Minimal Quick Start (Verify Link)

# Prepare Go environment (Linux x86_64)
go version || sudo apt-get update &amp;&amp; sudo apt-get install -y golang

# Get and run rttys
git clone https://github.com/zhaojh329/rttys.git
cd rttys
go run main.go  # Default listens on :5912
# Access in browser: http://&lt;serverIP&gt;:5912 to confirm the page opens

Managed by systemd (daemon/auto-start/logging)

# /etc/systemd/system/rttys.service
[Unit]
Description=rttys server (WebSocket terminal broker)
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
WorkingDirectory=/opt/rttys
ExecStart=/usr/local/bin/rttys
Restart=always
RestartSec=3
User=nobody
Group=nogroup
LimitNOFILE=65536
Environment=RTTYS_ADDR=:5912

[Install]
WantedBy=multi-user.target
# Install and enable
install -d /opt/rttys
cp ./rttys /usr/local/bin/rttys  # If using the binary generated by go build
systemctl daemon-reload
systemctl enable --now rttys
systemctl status rttys -n 50 --no-pager

Notes:

  • • It is recommended to manage the listening address through the environment variable <span>RTTYS_ADDR=:5912</span>, to avoid hardcoding.
  • • <span>Restart=always</span> + <span>RestartSec=3</span> ensures the process is automatically restarted after unexpected exits.

Enable TLS and Reverse Proxy

In production environments, it is recommended to use Nginx/Caddy/Traefik to provide HTTPS and correctly handle WebSocket upgrades:

# /etc/nginx/conf.d/rttys.conf
server {
    listen 443 ssl;
    server_name rtty.example.com;

    ssl_certificate     /etc/ssl/certs/fullchain.pem;
    ssl_certificate_key /etc/ssl/private/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:5912;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_read_timeout 3600s;
    }

    # Optional: Simple basic authentication
    # auth_basic           "Restricted";
    # auth_basic_user_file /etc/nginx/.htpasswd;
}
nginx -t &amp;&amp; systemctl reload nginx

Network and Firewall:

  • • Allow inbound port on the server:<span>443</span> (if directly connecting to rttys, then allow <span>5912</span>).
  • • The client only needs to connect to the server on that port (commonly device → cloud outbound 443).

Client rtty Deployment on Embedded Devices

Dependencies and Compilation Options

Client dependencies:

  • • <span>libev</span> (event loop)
  • • <span>libuwsc</span> (lightweight WebSocket client)
  • • SSL library (choose one):<span>mbedtls</span>/<span>wolfssl</span>/<span>openssl</span> (recommended to enable for wss support)

General cross-compilation approach:

export TOOLCHAIN_PREFIX=arm-linux-gnueabihf-
export SYSROOT=/opt/toolchains/arm-sysroot
export CC=${TOOLCHAIN_PREFIX}gcc
export PKG_CONFIG_PATH=$SYSROOT/usr/lib/pkgconfig:$SYSROOT/usr/share/pkgconfig
export PKG_CONFIG_SYSROOT_DIR=$SYSROOT

Build dependencies (example with openssl + libev + libuwsc):

# Cross-compile from source (example command, adjust according to actual version and path)

# OpenSSL (or use mbedtls/wolfssl)
tar xf openssl-1.1.1w.tar.gz &amp;&amp; cd openssl-1.1.1w
./Configure linux-generic32 --prefix=/usr --cross-compile-prefix=${TOOLCHAIN_PREFIX} --openssldir=/etc/ssl
make -j &amp;&amp; make DESTDIR="$SYSROOT" install

# libev
tar xf libev-4.33.tar.gz &amp;&amp; cd libev-4.33
./configure --host=arm-linux --prefix=/usr --with-pic --disable-shared --enable-static
make -j &amp;&amp; make DESTDIR="$SYSROOT" install

# libuwsc (supports pkg-config)
git clone https://github.com/zhaojh329/libuwsc.git &amp;&amp; cd libuwsc
make CC=${CC} SYSROOT=${SYSROOT} PREFIX=/usr STATIC=1
make DESTDIR="$SYSROOT" PREFIX=/usr install

Compile rtty client:

git clone https://github.com/zhaojh329/rtty.git
cd rtty

# CMake build (recommended)
cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_SYSROOT=$SYSROOT \
  -DCMAKE_C_COMPILER=${CC} \
  -DCMAKE_FIND_ROOT_PATH=$SYSROOT \
  -DOPENSSL_ROOT_DIR=$SYSROOT/usr \
  -DMBEDTLS_ROOT_DIR=$SYSROOT/usr

cmake --build build -j
file build/rtty &amp;&amp; ${TOOLCHAIN_PREFIX}strip build/rtty

Copy to device:

scp build/rtty root@&lt;device-ip&gt;:/usr/bin/rtty
ssh root@&lt;device-ip&gt; 'chmod +x /usr/bin/rtty'

Run and Auto-Start

Minimal run:

/usr/bin/rtty -I &lt;deviceID&gt; -h &lt;rttys server address or domain name&gt;
# For example: /usr/bin/rtty -I gw-7f3a2c -h rtty.example.com

systemd (for systemd devices):

# /etc/systemd/system/rtty.service
[Unit]
Description=rtty client (connects device terminal to rttys)
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
Environment=DEVICE_ID=gw-7f3a2c
Environment=RTTYS_HOST=rtty.example.com
ExecStart=/usr/bin/rtty -I ${DEVICE_ID} -h ${RTTYS_HOST}
Restart=always
RestartSec=2

[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable --now rtty
systemctl status rtty -n 50 --no-pager

init.d (BusyBox/non-systemd devices):

#!/bin/sh
# /etc/init.d/rtty

### BEGIN INIT INFO
# Provides:          rtty
# Required-Start:    $network
# Required-Stop:     $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
### END INIT INFO

DAEMON=/usr/bin/rtty
ID_FILE=/etc/rtty/device_id
HOST_FILE=/etc/rtty/host

[ -x "$DAEMON" ] || exit 0

start() {
    mkdir -p /etc/rtty
    [ -f "$ID_FILE" ] || echo gw-7f3a2c &gt; "$ID_FILE"
    [ -f "$HOST_FILE" ] || echo rtty.example.com &gt; "$HOST_FILE"
    ID=$(cat "$ID_FILE")
    HOST=$(cat "$HOST_FILE")
    start-stop-daemon -S -b -x "$DAEMON" -- -I "$ID" -h "$HOST"
}

stop() {
    start-stop-daemon -K -x "$DAEMON"
}

case "$1" in
  start) start ;; 
  stop)  stop  ;;
  restart) stop; start ;;
  *) echo "Usage: $0 {start|stop|restart}" ; exit 1 ;;
esac
exit 0
chmod +x /etc/init.d/rtty &amp;&amp; /etc/init.d/rtty start

Troubleshooting Common Issues

  1. 1. The browser can open the page but the connection drops instantly
  • • Check if the reverse proxy is correctly set up with <span>Upgrade/Connection</span> headers, and ensure <span>proxy_read_timeout</span> is set sufficiently high.
  • • Large time drift between server/device causing TLS handshake failure (enable NTP).
  1. 2. The device reports missing libraries when running
  • • <span>ldd /usr/bin/rtty</span> to check if dependencies are in the target root filesystem; if necessary, switch to static linking or copy missing <span>.so</span> files.
  1. 3. The device can ping the domain but cannot connect to the port
  • • <span>telnet rtty.example.com 443</span> or <span>nc -vz rtty.example.com 443</span> to verify that outbound traffic is allowed by the firewall.
  1. 4. Operation delays after many devices connect simultaneously
  • • Check rttys server CPU/memory and file handle limits; increase <span>LimitNOFILE</span>, add more instances, and implement load balancing.
  1. 5. Occasional disconnections
  • • Increase the timeout and buffering of the reverse proxy, check for link jitter (cellular/weak network); use system services to auto-restart the client.

Log location:

# Server
journalctl -u rttys -n 200 --no-pager

# Client (systemd devices)
journalctl -u rtty  -n 200 --no-pager

Leave a Comment