Kubernetes (K8s) is an open-source container orchestration platform developed by Google that automates the deployment, scaling, and management of containerized applications. It provides features such as high availability, elastic scaling, service discovery, rolling updates, and self-healing, freeing operations teams to focus on business, making it the standard infrastructure for the cloud-native era.
PreparationThis article is aimed at beginners and users trying out new versions, using a single-node demonstration, with plans to update to a high-availability version in the future.The operating system is Rocky Linux 10.The installation tool used is the official kubeadm.
| Hostname | IP | Remarks |
| k8s-master | 192.168.1.3 | Control node; it is recommended to have an odd number of nodes (3, 5, 7) in production environments. |
| k8s-worker1 | 192.168.1.4 | Worker node; the node that actually runs the containers. |
| k8s-worker2 | 192.168.1.5 | Worker node. |
Network Segment Planning
| Network Segment | Remarks |
| 192.168.1.0/24 | Network segment for inter-node communication, used for creating the cluster and scheduling (actual IP addresses of the hosts). |
| 10.244.0.0/16 | Pod network segment, container address range. |
| 10.96.0.0/12 | Service network segment, virtual IP range for services. |
Preliminary Preparation1. Change the hostname (on all nodes).
hostnamectl set-hostname k8s-master# hostnamectl set-hostname k8s-worker1# hostnamectl set-hostname k8s-worker22. Change the host IP to a static address (on all nodes).
# Change IP, network card name: ens160nmcli connection modify ens160 ipv4.method manual ipv4.addresses 192.168.1.3/24 ipv4.gateway 192.168.1.2 ipv4.dns 114.114.114.114# Restart the networknmcli connection down ens160 && nmcli connection up ens1603. Change the timezone (on all nodes).
# Set timezone (on all nodes)timedatectl set-timezone Asia/Shanghai# 24-hour formatlocalectl set-locale LC_TIME=en_GB.UTF-84. Node time synchronization (optional, required in production environments).
Install the chrony server (on the master node).
# Time synchronization server, usually the master node acts as this roleyum install chrony -ycat > /etc/chrony.conf << EOF # Sync from public pool ntp.aliyun.com iburst# Specify using ntp.aliyun.com as the time server pool, the iburst option means multiple requests will be sent during initial synchronization to speed up the processdriftfile /var/lib/chrony/drift# When the system time deviates from the server time by more than 1 second, it will adjust in 1-second increments. If the deviation exceeds 3 seconds, it will immediately adjust the time.makestep 1.0 3# Enable hardware clock synchronization to improve clock accuracy.rtcsync# Allow hosts in the 10.20.13.0/24 range to synchronize time with chrony.allow 0.0.0.0/24# Set the local clock to stratum 10, where the stratum value indicates clock accuracy; the lower the value, the higher the accuracy.local stratum 10# Specify the path to the key file used for time synchronization authentication.keyfile /etc/chrony.keys# Set timezone to UTCleapsectz right/UTC# Specify the log file storage directorylogdir /var/log/chronyEOF# Restart chronyd systemctl restart chronyd ; systemctl enable chronydInstall the chrony client (on the other nodes).
# Client, other nodes operationyum install chrony -ycat > /etc/chrony.conf << EOF # Sync from server pool 192.168.1.3 iburst# Specify using ntp.aliyun.com as the time server pool, the iburst option means multiple requests will be sent during initial synchronization to speed up the processdriftfile /var/lib/chrony/drift# When the system time deviates from the server time by more than 1 second, it will adjust in 1-second increments. If the deviation exceeds 3 seconds, it will immediately adjust the time.makestep 1.0 3# Enable hardware clock synchronization to improve clock accuracy.rtcsync# Specify the path to the key file used for time synchronization authentication.keyfile /etc/chrony.keys# Set timezone to UTCleapsectz right/UTC# Specify the log file storage directorylogdir /var/log/chronyEOF# Restart chronyd systemctl restart chronyd ; systemctl enable chronyd# Validate using the clientchronyc sources -v5. Disable the firewall (on all nodes).
# In Rocky 10, iptables has been replaced by firewalld, just disable firewalld# iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X && iptables -P FORWARD ACCEPTsystemctl disable firewalld --now6. Disable SELinux (on all nodes).
# Disable selinuxsetenforce 0sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/g' /etc/selinux/config7. Disable swap (on all nodes).
# Temporarily disable swapswapoff -a# Permanently disable swapsed -i 's/.*swap.*/#&/g' /etc/fstab8. Install basic system tools (optional).
yum update -yyum -y install wget psmisc jq vim net-tools nfs-utils telnet yum-utils device-mapper-persistent-data lvm2 git tar curl9. Configure ulimit, adjust the maximum number of open files in Linux (on all nodes).
ulimit -SHn 65535cat > /etc/security/limits.conf <<EOF# soft indicates soft limit, nofile indicates the maximum number of files a process can open, default is 1024. Here the soft limit is set to 655360, meaning a process can open a maximum of 655360 files.* soft nofile 655360# hard indicates hard limit, the maximum value set by the system. nofile indicates the maximum number of files a process can open, default is 4096. Here the hard limit is set to 131072, meaning the maximum number of files set by the system is 131072.* hard nofile 131072# nproc indicates the maximum number of processes a user can create, default is 30720. Here the maximum number of processes a user can create is set to 655350.* soft nproc 655350# nproc indicates the maximum number of processes a user can create, default is 4096. Here the maximum number of processes set by the system is 655350.* hard nproc 655350# memlock indicates the maximum memory a process can lock in RAM, default is 64 KB. Here the soft limit is set to unlimited, meaning a process can lock an unlimited amount of memory.* soft memlock unlimited# memlock indicates the maximum memory a process can lock in RAM, default is 64 KB. Here the hard limit is set to unlimited, meaning the maximum memory locked by the system is unlimited.* hard memlock unlimiteddEOF10. Install the IPVS module (on all nodes).
yum install -y ipvsadm ipset sysstat conntrack libseccomp11. Automatically load kernel modules at boot (on all nodes).
cat > /etc/modules-load.d/k8s.conf <<EOF # IPVS is a module in the Linux kernel used for load balancing and high availability. It distributes incoming requests to backend servers on the frontend proxy server, providing high-performance and scalable network services.ip_vs# IPVS round-robin scheduling algorithmip_vs_rr# IPVS weighted round-robin scheduling algorithmip_vs_wrr# IPVS hash scheduling algorithmip_vs_sh# overlay is the default storage driver used by containerd, providing a lightweight, stackable, incremental file system. It creates a file system view for containers by overlaying file system layers on top of existing file systems. Each container can have its own set of file system layers, which can share files from the base image and be modified within the container. Using overlay can effectively utilize disk space and make containers lighter.overlay# Allow traffic on the Linux bridge to be processed by iptables, which Kubernetes network plugins (like Flannel, Calico) depend on.br_netfilter# nf_conntrack is used to track and manage network connections, including protocols like TCP, UDP, and ICMP. It is the basis for implementing firewall state tracking.nf_conntrack# ip_tables provides support for IP packet filtering and network address translation (NAT) functions in Linux systems.ip_tables# Extends the functionality of iptables, supporting more efficient IP address set operations.ip_set# Extends the functionality of iptables, supporting more efficient packet matching and operations.xt_set# User-space tools for configuring and managing the xt_set kernel module.ipt_set# Used for implementing reverse path filtering to prevent IP spoofing and DDoS attacks.ipt_rpfilter# Used to reject IP packets and send a response to the sender indicating the packet was rejected.ipt_REJECT# Used to implement IP encapsulation in IP (IP-over-IP) tunneling. It can create virtual tunnels to transmit IP packets between different networks.ipipEOF# Restart kernel load modules to immediately load modified kernel features.systemctl restart systemd-modules-load.service12. Modify kernel parameters, automatically set kernel parameters at boot (on all nodes).
cat > /etc/sysctl.d/k8s.conf << EOF# Enables IPv4 IP forwarding, allowing the server to forward packets as a network router.net.ipv4.ip_forward = 1# When using network bridging technology, packets are passed to iptables for processing.net.bridge.bridge-nf-call-iptables = 1# When this parameter is set to 1, IPv6 packets will be passed to ip6tables for processing; when set to 0, IPv6 packets will bypass ip6tables and be passed directly. By default, this parameter is set to 1.net.bridge.bridge-nf-call-ip6tables = 1# Allow mounts to be used by other processes when mounting file systems.fs.may_detach_mounts = 1# Allow the original memory overcommit policy, where the system will still allocate extra memory when the system's memory is fully used.vm.overcommit_memory=1# Disable system crash and restart on out-of-memory (OOM) situations.vm.panic_on_oom=0# Set the maximum number of files a user can monitor with inotify instances.fs.inotify.max_user_watches=89100# Set the maximum number of files that can be opened simultaneously by the system.fs.file-max=52706963# Set the maximum number of open file descriptors that can exist simultaneously.fs.nr_open=52706963# Set the maximum number of network connection tracking table entries that can be created.net.netfilter.nf_conntrack_max=2310720# Set the idle timeout for TCP sockets (seconds); if no active data is received after this time, the kernel will send heartbeat packets.net.ipv4.tcp_keepalive_time = 600# Set the number of TCP heartbeat probes to send without receiving a response.net.ipv4.tcp_keepalive_probes = 3# Set the interval (seconds) for TCP heartbeat probes.net.ipv4.tcp_keepalive_intvl =15# Set the maximum number of TIME_WAIT sockets that can be used by the system.net.ipv4.tcp_max_tw_buckets = 36000# Enable reuse of TIME_WAIT sockets, allowing new sockets to use old TIME_WAIT sockets.net.ipv4.tcp_tw_reuse = 1# Set the maximum number of TCP socket garbage collection packets that can exist simultaneously.net.ipv4.tcp_max_orphans = 327680# Set the number of retries for isolated TCP sockets.net.ipv4.tcp_orphan_retries = 3# Enable TCP SYN cookies protection to prevent SYN flood attacks.net.ipv4.tcp_syncookies = 1# Set the maximum length of the half-open connection queue for new TCP connections.net.ipv4.tcp_max_syn_backlog = 16384# Set the maximum number of network connection tracking table entries that can be created.net.ipv4.ip_conntrack_max = 65536# Disable TCP timestamp functionality for better security.net.ipv4.tcp_timestamps = 0# Set the maximum value for the connection queue in the core layer of the system.net.core.somaxconn = 16384# Enable IPv6 protocol.net.ipv6.conf.all.disable_ipv6 = 0# Enable IPv6 protocol.net.ipv6.conf.default.disable_ipv6 = 0# Enable IPv6 protocol.net.ipv6.conf.lo.disable_ipv6 = 0# Allow IPv6 packet forwarding.net.ipv6.conf.all.forwarding = 1EOF# Reload kernel modules to make kernel parameters effective.sysctl --systemSupplement:
| Command | Operation | Impact |
| sysctl –system | Only set kernel parameters | /etc/sysctl.d/*.conf/etc/sysctl.conf |
| systemctl restart systemd-modules-load.service | Only load kernel modules | <span><span>/etc/modules-load.d/*.conf</span></span> |
13. Install Containerd (on all nodes).
# Install system toolsyum install -y yum-utils# Step 2: Add software source informationyum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo# Install containerd and crictl toolsyum -y install containerd cri-tools# Enable and start at boot.systemctl enable containerd --now# Check version informationcontainerd -v14. Modify Containerd configuration file (on all nodes).
mkdir -p /etc/containerd# Generate default configuration filecontainerd config default > /etc/containerd/config.toml# The SystemdCgroup parameter ensures that containerd can correctly manage container resource usage for resource limits, isolation, and fair distribution.sed -i "s#SystemdCgroup\ \= false#SystemdCgroup\ \= true#g" /etc/containerd/config.toml# Change the image pull address to a domestic address, here is the pause image address.sed -i "s#registry.k8s.io#registry.aliyuncs.com/google_containers#g" /etc/containerd/config.toml# Change pause version.sed -i 's/pause:3.6/pause:3.9/g' /etc/containerd/config.tomlSet Containerd image acceleration (on all nodes).
# Specify configuration file directorysed -i "s#config_path\ \= \"\"#config_path\ \=\ "/etc/containerd/registry"#g" /etc/containerd/config.tomlmkdir /etc/containerd/registry/docker.io -pcat > /etc/containerd/registry/docker.io/hosts.toml << EOFserver = "https://docker.io"[host."https://docker.1panel.live"] capabilities = ["pull","resolve","push"][host."https://docker.1ms.run"] capabilities = ["pull","resolve","push"] [host."https://docker.m.daocloud.io"] capabilities = ["pull", "resolve"][host."https://xk9ak4u9.mirror.aliyuncs.com"] capabilities = ["pull","resolve"][host."https://dockerproxy.com"] capabilities = ["pull", "resolve"][host."https://docker.mirrors.sjtug.sjtu.edu.cn"] capabilities = ["pull","resolve"][host."https://docker.mirrors.ustc.edu.cn"] capabilities = ["pull","resolve"][host."https://docker.nju.edu.cn"] capabilities = ["pull","resolve"][host."https://registry-1.docker.io"] capabilities = ["pull","resolve","push"]EOF
mkdir /etc/containerd/registry/gcr.io -pcat > /etc/containerd/registry/gcr.io/hosts.toml << EOFserver = "https://gcr.io"[host."https://gcr.m.daocloud.io"] capabilities = ["pull", "resolve"]EOF
mkdir /etc/containerd/registry/registry.k8s.io -pcat > /etc/containerd/registry/registry.k8s.io/hosts.toml << EOFserver = "https://registry.k8s.io"[host."https://k8s.m.daocloud.io"] capabilities = ["pull", "resolve"]EOF
mkdir /etc/containerd/registry/k8s.gcr.io -pcat > /etc/containerd/registry/k8s.gcr.io/hosts.toml << EOFserver = "https://k8s.gcr.io"[host."https://k8s.m.daocloud.io"] capabilities = ["pull", "resolve"]EOF
mkdir /etc/containerd/registry/quay.io -pcat > /etc/containerd/registry/quay.io/hosts.toml << EOFserver = "https://quay.io"[host."https://quay.m.daocloud.io"] capabilities = ["pull", "resolve"]EOF# Reload configuration and restart.systemctl daemon-reload && systemctl restart containerd# Set to start automatically at boot and start.systemctl enable containerd --now15. Configure crictl (optional).
cat > /etc/crictl.yaml <<EOFruntime-endpoint: unix:///var/run/containerd/containerd.sockimage-endpoint: unix:///var/run/containerd/containerd.socktimeout: 10debug: falsepull-image-on-create: falseEOF16. Install k8s packages (on all nodes).
# Specify the version of k8s tools.export k8sVersion=v1.34# Write to yum sourcecat > /etc/yum.repos.d/kubernetes.repo << EOF[kubernetes]name=Kubernetesbaseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/$k8sVersion/rpm/enabled=1gpgcheck=1gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/$k8sVersion/rpm/repodata/repomd.xml.keyEOF# Check available versionsyum list --showduplicates --disableexcludes=kubernetes | grep kubeadm# Installyum install -y kubelet-1.34.0 kubeadm-1.34.0 kubectl-1.34.0systemctl enable kubelet --nowWorker nodes only need to install kubelet and kubeadm.
yum install -y kubelet-1.34.0 kubeadm-1.34.0systemctl enable kubelet --now17. Create kubeadm.yml file.
kubeadm config print init-defaults --component-configs KubeletConfiguration > kubeadm.ymlExplanation:
# Use systemd cgroup--component-configs KubeletConfigurationEdit kubeadm.yml (modify comment positions).
apiVersion: kubeadm.k8s.io/v1beta4bootstrapTokens:- groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages: - signing - authenticationkind: InitConfigurationlocalAPIEndpoint: advertiseAddress: 192.168.1.3 # Local host IP bindPort: 6443nodeRegistration: criSocket: unix:///var/run/containerd/containerd.sock imagePullPolicy: IfNotPresent imagePullSerial: true name: k8s-master # Local host name taints: nulltimeouts: controlPlaneComponentHealthCheck: 4m0s discovery: 5m0s etcdAPICall: 2m0s kubeletHealthCheck: 4m0s kubernetesAPICall: 1m0s tlsBootstrap: 5m0s upgradeManifests: 5m0s---apiServer: {}apiVersion: kubeadm.k8s.io/v1beta4caCertificateValidityPeriod: 87600h0m0scertificateValidityPeriod: 8760h0m0scertificatesDir: /etc/kubernetes/pkiclusterName: kubernetescontrollerManager: {}dns: {}encryptionAlgorithm: RSA-2048etcd: local: dataDir: /var/lib/etcdimageRepository: registry.aliyuncs.com/google_containers # Use Alibaba Cloud image sourcekind: ClusterConfigurationkubernetesVersion: 1.34.0 # Version number, must correspond to installed packagesnetworking: dnsDomain: cluster.local podSubnet: 10.244.0.0/16 # New pod subnet serviceSubnet: 10.96.0.0/12proxy: {}scheduler: {}---apiVersion: kubelet.config.k8s.io/v1beta1authentication: anonymous: enabled: false webhook: cacheTTL: 0s enabled: true x509: clientCAFile: /etc/kubernetes/pki/ca.crtauthorization: mode: Webhook webhook: cacheAuthorizedTTL: 0s cacheUnauthorizedTTL: 0scgroupDriver: systemd # Use systemd cgroupclusterDNS:- 10.96.0.10clusterDomain: cluster.localcontainerRuntimeEndpoint: ""cpuManagerReconcilePeriod: 0scrashLoopBackOff: {}evictionPressureTransitionPeriod: 0sfileCheckFrequency: 0shealthzBindAddress: 127.0.0.1healthzPort: 10248httpCheckFrequency: 0simageMaximumGCAge: 0simageMinimumGCAge: 0skind: KubeletConfigurationlogging: flushFrequency: 0 options: json: infoBufferSize: "0" text: infoBufferSize: "0" verbosity: 0memorySwap: {}nodeStatusReportFrequency: 0snodeStatusUpdateFrequency: 0srotateCertificates: trueruntimeRequestTimeout: 0sshutdownGracePeriod: 0sshutdownGracePeriodCriticalPods: 0sstaticPodPath: /etc/kubernetes/manifestsstreamingConnectionIdleTimeout: 0ssyncFrequency: 0svolumeStatsAggPeriod: 0s---# kubeproxy uses IPVS mode# Reference: https://github.com/kubernetes/kubernetes/blob/release-1.34/pkg/proxy/ipvs/README.mdapiVersion: kubeproxy.config.k8s.io/v1alpha1kind: KubeProxyConfigurationmode: ipvs18. Initialize the cluster (master node operation).
# Pre-pull imageskubeadm config images pull --config kubeadm.yml# Initialize the clusterkubeadm init phase upload-certs --upload-certs# --upload-certs is generally used in multi-master setups to automatically distribute certificates without manually copying /etc/kubernetes/pki, which is not covered in this instance.Supplement:
# Validate if the configuration file is compliantkubeadm config validate --config kubeadm.yml
# View the list of required imageskubeadm config images list --config kubeadm.yml
# Migrate old kubeadm file to a supported kubeadm.ymlkubeadm config migrate --old-config kubeadm.yml --new-config new-kubeadm.yml
# Generate join node commandkubeadm token create --print-join-command
# Remove taints as needed, allowing containers to run on the master.kubectl taint node --all node-role.kubernetes.io/control-plane-19. Configure kubelet configuration file.
# Configure kubectlmkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/config20. Configure command auto-completion (optional, recommended, master node operation).
yum install bash-completion -ysource /usr/share/bash-completion/bash_completionsource <(kubectl completion bash)echo "source <(kubectl completion bash)" >> ~/.bashrc21. Join the cluster (worker node operation).
kubeadm join 192.168.1.3:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>Verify nodes
kubectl get nodeNAME STATUS ROLES AGE VERSIONk8s-master NotReady control-plane 42s v1.34.0k8s-worker1 NotReady <none> 8s v1.34.0k8s-worker2 NotReady <none> 5s v1.34.0The cluster has been successfully created, and the nodes have joined, but the status is still NotReady. In the next article, we will install the network components to enable container communication.