Background

Open source is the cornerstone of innovation (whether acknowledged or not). The foundation of the innovation operating system is Linux. The use, tuning, and troubleshooting of Linux require many tools.

Basic Tools

Linux connections: Finalshell, Winterm, MobaXterm, Xshell, and Windows bash. Open source tools can be used: winscp for file uploads, putty for system connections.

Basic Commands – Those with some foundation can skip this section

Note: Most basic commands are in lowercase. Linux is case-sensitive. cd to switch to a directory, cd to switch to the user's home directory, cd - to switch to the previous directory. ls to display files/directories in the directory, ls -l to show detailed information, ls -al to show hidden files. su to switch users, e.g., su - oracle where '-' represents the target user's environment variables. ip addr to display the IP address. hostname to display the machine name. hostnamectl set-hostname newname to change the name. tar for packaging/unpacking commands. -czvf to create a gzip compressed package, -zxvf to decompress a gzip compressed file. unzip to decompress zip files, add -O CP936 to decompress according to the GBK character set to avoid Chinese garbled text. mkdir to create directories. mkdir -p to create directories recursively and avoid errors if directories already exist. rm -rf if the path is '/' then it is an automatic resignation command. rm -rf must be used with caution, it will recursively delete directory files. ./some.sh to execute the shell script or command some.sh in the current directory. nohup ./some.sh >/dev/null && to run commands in the background. Avoid remote termination and service interruption. cat to view files or merge multiple files into one. tail -f to dynamically view files. When the file changes, it can be viewed directly. chmod to modify file permissions, note that it cannot be executed under /, it can be executed in specific directories. chown to change file ownership. Generally required when databases or non-root users execute software.

Advanced Commands 0.1

The three musketeers of Linux text: sed for stream processing files, grep for filtering text, awk for processing and displaying text. Important command tools: vim for command-line file processing. vim command interface -> editing interface -> visual interface. It is necessary to memorize the commands inside for convenient text modification. Red Hat software installation: yum install/rpm -ivh for online/offline installation packages. top to view CPU usage, you can input P to sort by CPU, input M to sort by process memory. top -Hp $pid can view multiple threads' usage under a single process. top -Hp $pid -bn 1 can print the results. You can use > to write to a specific file. ps -ef |grep java can view Java process information. free -m to view the remaining memory information of the operating system in MB. df -Th to check disk capacity. du -ahd 1 |sort -k1h to view the size of data files in the directory. lscpu to view CPU core count/architecture/base frequency information.

Advanced Commands 0.2

For Red Hat systems, install yum install sysstat. Command breakdown: sar collects and displays all system activity statistics. sadc stands for "System Activity Data Collector". This is the backend tool for data collection of sar. sa1 stores system activity in binary data files. For this purpose, sa1 relies on sadc. sa1 runs from cron. sa2 creates a daily summary of collected statistics. sa2 runs from cron. sadf can generate CSV, XML, and various other formats of sar reports. Use it to integrate sar data with other tools. iostat generates CPU and I/O statistics. mpstat displays CPU statistics. pidstat reports statistics based on process ID (PID). nfsiostat displays NFS I/O statistics. cifsiostat generates CIFS statistics.

Advanced Commands 0.2

sar displays CPU information. sar --help displays help information. sar -r displays memory-related information. sar -B displays memory swap information. sar -b displays IO-related information. sar -n DEV ens60 displays network card information. sar is actually more useful than node exporter. In the absence of monitoring, sar is the most effective weapon for problem diagnosis.

Combined Command 1

for i in `ps -ef |grep java |grep -v nacos |grep -v grep |awk '{print $2}'` ; do echo $i ;  echo -n "Total heap memory for this process (KB): " ; pmap -x $i |sort -k3h |tail -n 2 |cut -c 17- |awk '{print $2}' |head -n 1 ; pmap -x $i |sort -k3h |tail -n 2 |cut -c 17- |awk '{print $2}' |awk 'NR==2{second=$1} NR==1{first=$1; next} END{print "Total off-heap memory for this process (KB): " second-first}' ; echo -n "Total memory for this process (KB): " ; pmap -x $i |sort -k3h |tail -n 2 |cut -c 17- |awk '{print $2}' |tail -n 1 ; done

Effect of Combined Command 1

Check the memory information of non-nacos Java processes on the current server. It is mainly divided into three parts: print the thread ID followed by the heap memory, off-heap memory, and total memory used by the process. The effect is: 181248 Total heap memory for this process (KB): 1167872 Total off-heap memory for this process (KB): 970148 Total memory for this process (KB): 2138020

Simple Explanation of Combined Command 1

for i in `ps -ef |grep java |grep -v nacos |grep -v grep |awk '{print $2}'` to view Java process information that is not nacos and not the current grep process. And only get the PID information of these processes. Then use a for loop to assign values. do echo $i ; echo -n "Total heap memory for this process (KB): " ; print this PID and do not print a newline for "Total heap memory for this process." pmap -x $i |sort -k3h |tail -n 2 |cut -c 17- |awk '{print $2}' |head -n 1 ; get the memory information of this process through pmap, sort by RSS, and only take the smaller one of the maximum two. pmap -x $i |sort -k3h |tail -n 2 |cut -c 17- |awk '{print $2}' |awk 'NR==2{second=$1} NR==1{first=$1; next} END{print "Total off-heap memory for this process (KB): " second-first}' ; take these two for subtraction, which is the non-heap area information in pmap monitoring. echo -n "Total memory for this process (KB): " ; pmap -x $i |sort -k3h |tail -n 2 |cut -c 17- |awk '{print $2}' |tail -n 1 ; done The maximum line of data is the memory information used by this process. Note that this information is the same as the RSS value in the top command. It can also be understood that top retrieves data from pmap.

Combined Command 2

cat > checkprocess.sh << EOF let uptime=`cat /proc/sched_debug |grep ktime |awk '{print $3}' |awk -F '.' '{print $1}'` let uptime=uptime/10 let pid=`ps -ef |grep java |grep caf |head -n 1 |awk '{print $2}'` for i in `ls /proc/$pid/task` ; do echo -n `cat /proc/$pid/task/$i/stat |awk -F '(' '{print $2}' |awk -F ')' '{print $1}'` ; echo -n " Thread ID: " echo -n `cat /proc/$pid/task/$i/stat |awk '{print $1}'` ; echo -n " Time after thread startup (seconds): " let threadtime=`cat /proc/$pid/task/$i/stat | awk -F ')' '{print $2}' |awk '{print $20}'` ; let threadtime=(uptime-threadtime)/100 ; echo -n $threadtime echo -n " CPU time used by the thread (milliseconds): " let runingtime=`cat /proc/$pid/task/$i/sched |grep se.sum_exec_runtime |awk '{print $3}' |awk -F '.' '{print $1}'` ; let runingtime=runingtime echo $runingtime done EOF

Effect of Combined Command 2

chmod 777 checkprocess.sh ./checkprocess.sh The result is: java Thread ID: 20845 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 3 java Thread ID: 20850 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 574998 java Thread ID: 20867 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 6965 java Thread ID: 20868 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 7020 java Thread ID: 20869 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 6984 java Thread ID: 20870 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 6911 java Thread ID: 20871 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 6977 java Thread ID: 20872 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 6929 java Thread ID: 20873 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 6886 java Thread ID: 20874 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 6944 java Thread ID: 20875 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 7021 java Thread ID: 20876 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 7005 VM Thread Thread ID: 20881 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 1793 Reference Handler Thread ID: 20885 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 123 Finalizer Thread ID: 20887 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 160 Signal Dispatch Thread ID: 20906 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 0 server-timer Thread ID: 21005 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 5 server-timer1 Thread ID: 21006 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 39 Thread-3 Thread ID: 21017 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 71 C2 Compiler Thread ID: 21018 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 14623 C2 Compiler Thread ID: 21019 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 15897 C2 Compiler Thread ID: 21020 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 13619 C1 Compiler Thread ID: 21021 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 3457 Service Thread ID: 21023 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 70 VM Periodic Task Thread ID: 21025 Time after thread startup (seconds): 586 CPU time used by the thread (milliseconds): 461 prometheus-http Thread ID: 21148 Time after thread startup (seconds): 585 CPU time used by the thread (milliseconds): 1355 logback-1 Thread ID: 21177 Time after thread startup (seconds): 579 CPU time used by the thread (milliseconds): 2 logback-2 Thread ID: 21805 Time after thread startup (seconds): 519 CPU time used by the thread (milliseconds): 1 logback-3 Thread ID: 22328 Time after thread startup (seconds): 459 CPU time used by the thread (milliseconds): 0 pool-3-thread-8 Thread ID: 22736 Time after thread startup (seconds): 409 CPU time used by the thread (milliseconds): 49827 logback-4 Thread ID: 22806 Time after thread startup (seconds): 399 CPU time used by the thread (milliseconds): 0 logback-5 Thread ID: 23284 Time after thread startup (seconds): 339 CPU time used by the thread (milliseconds): 0 logback-6 Thread ID: 23758 Time after thread startup (seconds): 279 CPU time used by the thread (milliseconds): 0 pool-3-thread-1 Thread ID: 23942 Time after thread startup (seconds): 252 CPU time used by the thread (milliseconds): 21783 logback-7 Thread ID: 24235 Time after thread startup (seconds): 219 CPU time used by the thread (milliseconds): 0 pool-3-thread-1 Thread ID: 24448 Time after thread startup (seconds): 188 CPU time used by the thread (milliseconds): 31017 pool-3-thread-1 Thread ID: 24450 Time after thread startup (seconds): 188 CPU time used by the thread (milliseconds): 28568 logback-8 Thread ID: 24759 Time after thread startup (seconds): 159 CPU time used by the thread (milliseconds): 0