Getting Started with OpenYurt on Raspberry Pi

Author | Tang Bingchang

With the rapid development of edge computing, more and more data needs to be stored, processed, and analyzed at the edge of the network, leading to an explosive growth of edge devices and applications. Efficiently managing resources and applications at the edge is a major challenge faced by the industry. Currently, a cloud-native approach that integrates cloud computing capabilities into the edge and allows for unified scheduling and control in the cloud has gained widespread recognition in the industry.

In May 2020, Alibaba open-sourced the first Kubernetes non-intrusive edge computing cloud-native project, OpenYurt, which entered the CNCF SandBox in September of the same year. OpenYurt enhances native Kubernetes non-intrusively for edge scenarios, addressing issues such as unstable networks and difficult cloud-edge operations, focusing on providing capabilities for edge node autonomy, cloud-edge operation channels, and edge unitization.

As shown in Figure 1, this article demonstrates how to build a cloud management edge scenario by deploying a Kubernetes cluster control plane in the cloud and connecting a Raspberry Pi to the cluster. Based on this environment, we will showcase the core capabilities of OpenYurt and help you quickly get started with OpenYurt.

Figure 1 Native Kubernetes Cluster

Getting Started with OpenYurt on Raspberry Pi

Environment Preparation

1. Basic Environment Introduction

In the cloud, purchase an ENS node (ENS nodes have public IPs, making it easy to expose services to the public) to deploy the control components of the native K8s cluster. The system uses Ubuntu 18.04, with hostname set to master-node, and Docker version 19.03.5.

At the edge, as shown in Figure 2, connect Raspberry Pi 4 to the local router to form an edge private network environment, with the router accessing the internet through a 4G network card. The Raspberry Pi 4 comes pre-installed with Ubuntu 18.04, with hostname set to edge-node, and Docker version 19.03.5.

Figure 2 Edge Environment Physical Diagram

2. Building a Native K8s Cluster

This demonstration environment is based on the community version 1.16.6 of the K8s cluster and uses the kubeadm tool provided by the community to build the cluster. The specific operations are as follows:

Install Kubernetes components on the cloud node and Raspberry Pi by executing the following commands.

curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" &gt; /etc/apt/sources.list.d/kubernetes.listsudo apt-get updatesudo apt install -y kubelet=1.16.6-00 kubeadm=1.16.6-00 kubectl=1.16.6-00

Initialize the cloud node using kubeadm (execute the following command on the cloud node). During the deployment process, use Alibaba Cloud’s image repository. To support Raspberry Pi access, the images in this repository have a manifest list that can support both amd64 and arm64 CPU architectures.

# master-nodekubeadm init --image-repository=registry.cn-hangzhou.aliyuncs.com/edge-kubernetes --kubernetes-version=v1.16.6 --pod-network-cidr=10.244.0.0/16

According to the prompt after initialization, copy the config file to $HOME/.kube:

mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

Connect the Raspberry Pi to the cloud cluster according to the node connection information output after initialization in step two, and execute the connection command on the Raspberry Pi.

kubeadm join 183.195.233.42:6443 --token XXXX 
 --discovery-token-ca-cert-hash XXXX

Add CNI configuration (both the cloud control node and Raspberry Pi need to be configured). The cluster built in this article uses the host network. Create the CNI configuration file /etc/cni/net.d/0-loopback.conf and copy the following content into this file.

{ "cniVersion": "0.3.0", "name": "lo", "type": "loopback"}

Check the deployment effect on the master node.

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIMEedge-node Ready &lt;none&gt;   74s    v1.16.6   192.168.0.100    &lt;none&gt;        Ubuntu 18.04.4 LTS   4.19.105-v8-28      docker://19.3.5master-node   Ready    master   2m5s   v1.16.6   183.195.233.42   &lt;none&gt;        Ubuntu 18.04.2 LTS   4.15.0-52-generic   docker://19.3.5

Delete CoreDNS (CoreDNS is not needed in this demo) and remove the taints from the master node (to facilitate the subsequent deployment of OpenYurt components).

kubectl delete deployment coredns -n kube-systemkubectl taint node master-node node-role.kubernetes.io/master-

Getting Started with OpenYurt on Raspberry Pi

Issues with Native K8s Cluster in Edge Scenarios

Based on the above environment, we will test the support of native K8s for cloud-edge operations and its response to disconnections in the cloud-edge network. First, we will deploy a test application nginx from the cloud, executing kubectl apply -f nginx.yaml on the master node. The specific deployment yaml is as follows.

Note: nodeSelector selects the edge-node node, the host network configuration is set to true, and the pod’s toleration time is configured to 5s (default is 5min, this configuration is for demonstration purposes to evict the pod quickly).

apiVersion: v1kind: Podmetadata: name: nginxspec: tolerations: - key: "node.kubernetes.io/unreachable" operator: "Exists" effect: "NoExecute" tolerationSeconds: 5 - key: "node.kubernetes.io/not-ready" operator: "Exists" effect: "NoExecute" tolerationSeconds: 5 nodeSelector: kubernetes.io/hostname: edge-node containers: - name: nginx image: nginx hostNetwork: true

Check the deployment results:

root@master-node:~# kubectl get pods -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx 1/1 Running 0 11s 192.168.0.100 edge-node &lt;none&gt;           &lt;none&gt;

1. Test common cluster operation commands, including logs, exec, port-forward.

Perform operations on the edge node application from the master node, executing logs/exec/port-forward commands, and checking the results.

root@master-node:~# kubectl logs nginxError from server: Get https://192.168.0.100:10250/containerLogs/default/nginx/nginx: dial tcp 192.168.0.100:10250: connect: connection refused

root@master-node:~# kubectl exec -it nginx shkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.Error from server: error dialing backend: dial tcp 192.168.0.100:10250: connect: connection refused

root@master-node:~# kubectl port-forward pod/nginx 8888:80error: error upgrading connection: error dialing backend: dial tcp 192.168.0.100:10250: connect: connection refused

From the execution results, it can be seen that native K8s cannot provide the ability to operate edge applications from the cloud in the cloud-edge scenario. This is because the edge node is deployed in the user’s private network, and the cloud cannot directly access the edge node’s IP address.

2. Test the impact on business during edge disconnection

The edge node connects to the cloud control through the public network, and there are often unstable network conditions and disconnections from the cloud. Here we will conduct two disconnection-related tests:

Disconnect for 1 minute -> Restore network
Disconnect for 1 minute -> Restart edge node -> Restore network

Observe the status changes of nodes and Pods during the two tests. The disconnection method used in this demo is to disconnect the router’s public connection.

1）Disconnect for 1 minute -> Restore network

Disconnecting the network, after about 40 seconds, the node becomes NotReady (the normal node reports a heartbeat every 10 seconds, and when it fails to report for 4 times, the control component considers the node abnormal).

root@master-node:~# kubectl get nodesNAME STATUS ROLES AGE VERSIONedge-node NotReady &lt;none&gt;   5m13s   v1.16.6master-node   Ready      master   6m4s    v1.16.6

Continue to wait for 5 seconds (after a normal node becomes NotReady, it takes 5 minutes to evict pods; here, for testing purposes, the pod’s toleration time is set to 5 seconds), and the application pod is evicted, changing its status to Terminating.

root@master-node:~# kubectl get podsNAME READY STATUS RESTARTS AGEnginx 1/1 Terminating 0 3m45s

After restoring the network, observe the changes in nodes and pods.

root@master-node:~# kubectl get podsNo resources found in default namespace.

After the network is restored, the node status changes to ready, and the business pod is cleared. This is because the edge node’s Kubelet obtains the business pod’s Terminating status, performs a deletion operation on the business pod, and returns a successful deletion; the cloud also performs corresponding cleanup. Thus, the business pod was evicted due to the instability of the cloud-edge network, but during the disconnection period, the edge node could still operate normally.

Recreate the application nginx for the next test.

root@master-node:~# kubectl get pods -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx 1/1 Running 0 4s 192.168.0.100 edge-node &lt;none&gt;           &lt;none&gt;

2）Disconnect for 1 minute -> Restart edge node -> Restore network

Next, we will test the impact of restarting the edge node during a disconnection on the business. After 1 minute of disconnection, the Node and Pod status is the same as the previous test results, the Node becomes NotReady, and the Pod status remains Terminating. At this time, switch to the private network environment, log in to the Raspberry Pi, and restart it. After the restart, wait about 1 minute and observe the container list on the node before and after the restart.

Container list on the edge node before the restart (at this time the cloud-edge is connected, although the pod is in the Terminating state obtained from the cloud, the edge has not watched the Terminating operation, so the edge application is still running normally).

root@edge-node:~# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES9671cbf28ca6 e86f991e5d10 "/docker-entrypoint.…" About a minute ago Up About a minute k8s_nginx_nginx_default_efdf11c6-a41c-4b95-8ac8-45e02c9e1f4d_06272a46f93ef registry.cn-hangzhou.aliyuncs.com/edge-kubernetes/pause:3.1 "/pause" 2 minutes ago Up About a minute k8s_POD_nginx_default_efdf11c6-a41c-4b95-8ac8-45e02c9e1f4d_0698bb024c3db f9ea384ddb34 "/usr/local/bin/kube…" 8 minutes ago Up 8 minutes k8s_kube-proxy_kube-proxy-rjws7_kube-system_51576be4-2b6d-434d-b50b-b88e2d436fef_031952700c95b registry.cn-hangzhou.aliyuncs.com/edge-kubernetes/pause:3.1 "/pause" 8 minutes ago Up 8 minutes k8s_POD_kube-proxy-rjws7_kube-system_51576be4-2b6d-434d-b50b-b88e2d436fef_0

Container list on the edge node after the restart:

root@edge-node:~# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES0c66b87066a0 473ae979be68 "yurt-tunnel-agent -…" 12 seconds ago Up 11 seconds k8s_yurt-tunnel-agent_yurt-tunnel-agent-7l8nv_kube-system_75d28494-f577-43fa-9cac-6681a1215498_2a4fb3e4e8c8f e86f991e5d10 "/docker-entrypoint.…" 58 seconds ago Up 56 seconds k8s_nginx_nginx_default_b45baaac-eebc-466b-9199-2ca5c1ede9fd_1fce730d64b32 f9ea384ddb34 "/usr/local/bin/kube…" 58 seconds ago Up 57 seconds k8s_kube-proxy_kube-proxy-rjws7_kube-system_51576be4-2b6d-434d-b50b-b88e2d436fef_2c78166ea563f registry.cn-hangzhou.aliyuncs.com/edge-kubernetes/pause:3.1 "/pause" 59 seconds ago Up 57 seconds k8s_POD_yurt-tunnel-agent-7l8nv_kube-system_75d28494-f577-43fa-9cac-6681a1215498_1799ad14bcd3b registry.cn-hangzhou.aliyuncs.com/edge-kubernetes/pause:3.1 "/pause" 59 seconds ago Up 57 seconds k8s_POD_nginx_default_b45baaac-eebc-466b-9199-2ca5c1ede9fd_1627673da6a85 registry.cn-hangzhou.aliyuncs.com/edge-kubernetes/pause:3.1 "/pause" 59 seconds ago Up 58 seconds k8s_POD_kube-proxy-rjws7_kube-system_51576be4-2b6d-434d-b50b-b88e2d436fef_204da705e4120 70bf6668c7eb "yurthub --v=2 --ser…" About a minute ago Up About a minute k8s_yurt-hub_yurt-hub-edge-node_kube-system_d75d122e752b90d436a71af44c0a53be_1260057d935ee registry.cn-hangzhou.aliyuncs.com/edge-kubernetes/pause:3.1 "/pause" About a minute ago Up About a minute k8s_POD_yurt-hub-edge-node_kube-system_d75d122e752b90d436a71af44c0a53be_1

From the comparison before and after the restart, it can be seen that after the edge node is restarted during disconnection, the pods on the node can be started normally, and OpenYurt’s node autonomy capability can ensure stable operation of the business during disconnection.

After restoring the network, the node is Ready, and observe the status of the business pod. After the network is restored, the business pod status remains running, with one restart record, which is as expected.

root@master-node:~# kubectl get pods -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx 1/1 Running 1 11m 192.168.0.100 edge-node &lt;none&gt;           &lt;none&gt;

Finally, we use the capability of yurtctl to convert the OpenYurt cluster back to a native K8s cluster.Similarly, we can observe that there will be no impact on existing business during the conversion.

yurtctl revert --yurtctl-servant-image=registry.cn-hangzhou.aliyuncs.com/openyurt/yurtctl-servant:v0.2.1

OpenYurt, as Alibaba’s first edge cloud-native open-source project, is based on the commercial product ACK@Edge and has undergone extensive refinement within the group.It has been applied in numerous scenarios such as CDN, IoT, Hema, ENS, and Cainiao Logistics.For edge scenarios, this project maintains the characteristics of native K8s, providing capabilities such as edge node autonomy and integrated cloud-edge operation channels in the form of addons.Recently, with the efforts of community colleagues, the edge unitization management capability has been open-sourced, and more edge management capabilities will continue to be open-sourced in the future. We welcome everyone to actively participate and contribute.

We welcome everyone to participate in the co-construction of this project. If you have any questions, you can scan the QR code to join the group for communication.

Getting Started with OpenYurt on Raspberry Pi

Click the original text to learn more about the OpenYurt project!

1. Basic Environment Introduction

2. Building a Native K8s Cluster

1. Test common cluster operation commands, including logs, exec, port-forward.

2. Test the impact on business during edge disconnection

1）Disconnect for 1 minute -> Restore network

2）Disconnect for 1 minute -> Restart edge node -> Restore network

We welcome everyone to participate in the co-construction of this project. If you have any questions, you can scan the QR code to join the group for communication.

Related posts:

Leave a Comment Cancel reply