Emergency Response Procedures for Cybersecurity Incidents

Emergency Response Steps: 1. Focus on ensuring business stability.

Blacklist IP, rate limit, human verification code.

First, activate the operational emergency plan to ensure stable business operation and system stability (network, system files, startup items, libraries, memory). Experience is irreplaceable. Think about security incidents with an event correlation mindset. All operations must be recorded with timestamps.

1. Deploy security systems. Ensure all host clocks are synchronized. Server monitoring systems, system log systems, NIPS, WAF, HIDS, honeypots, host hardening, database auditing, NTA traffic visualization, code key leakage scanning, bastion host, vulnerability warning platform, SIEM.

2. Receive intrusion alerts from Anqi, honeypots, NIDS, DNS logs, external domain names, database auditing system with large amounts of log data, server crashes or reboots, customer service receives hacker information, server response speed is too slow, connections to the server outside working hours, abnormal network traffic, such as ping or scanning operations; a batch of servers is remotely linked, file integrity alarms, customer data loss, server finds files encrypted.

3. Information collection must be completed within 4 hours. Assess the impact on business, the name of the compromised project, general architecture, alarm information, number of affected hosts, hacker domain names, IPs, security system logs, trojan samples, whether the server can connect to the internet, whether there is a unified management entrance (bastion host), open ports, all hacker behavior records, operating system versions, patch status.

4. Intrusion analysis.

Use the intelligence threat platform to check URLs, virus files, IPs, domain names, locate hacker machines or the first intrusion point. Which address is the server linked to, hacker IP. Analyze the intrusion point, which machine was penetrated first. Check the attack source through honeypots. View logs from all security devices to locate the attack surface. Determine if the operation was executed on the office network machine. Through the group to which the server belongs, check for cloud account leaks. Virus analysis of all potentially attacked machines. Check if the preloaded library configuration files have been modified. Check if the dynamic link library configuration files have been modified. View system startup items. Use the company knowledge base to reference similar past cases.

1. Types of Security Incidents

Remote control, trojan implantation, backdoor, CPU spikes, abnormal traffic, rootkits, mining, ransomware, SQL database dumping, web penetration, black pages, web shell, cloud account leaks, insider theft, brute force attacks, credential leaks, office networks being compromised, network attacks, hijacking. Check all applications’ brute force logs and login logs, such as ssh and mysql, dns; kill malicious processes, check for dynamic link library backdoors.

1. Quick Emergency Response Plan

Add hacker IPs to the blacklist, directly implement DDos attacks, switch applications. Prohibit outgoing connections, block hacker remote control actions. Check external ports; if there are high-risk vulnerability applications, add them to the whitelist. Delete trojan files, kill processes; if rootkits are present, take the server offline. Identify machines that can still connect to the internet and focus on security hardening. Conduct deep security checks; take offline unimportant servers. Do not allow direct access to servers to prevent forward shells; use proxies instead. Add hacker IPs to the blacklist. Check external ports; if there are high-risk vulnerability applications, add them to the whitelist. Prohibit outgoing connections, block hacker remote control actions. Take machines offline. Check public vulnerabilities such as redis, mongodb, MySQL, etc. Do not allow direct access to servers to prevent forward shells; use proxies instead. Add hacker IPs to the blacklist. Add an additional layer of cloud WAF, only enable OWASP defense functions. Develop and modify interfaces to add verification codes for human recognition; limit access to the same interface to 10 times per minute. Find vulnerable interfaces through nginx logs. Check if there are trojans and web shells on the server. “1. Find the logs to see which machine generated them, find the keywords, 2. Go to the files to confirm if they are really injected with web shells; many are the hackers’ scanning logs. 3. Immediately delete, check which interface it came from, use the web shell content as keywords for a full search, especially Java logs. 4. Find all nginx logs, locate this request interface, use the ack command, which is better than grep. 5. Identify the specific interface and have the developers modify the filters.” Immediately modify cloud account, change keys, perform a remote connection to the cloud platform desktop; for the second time, phone verification is required. For sensitive operations and logins, enable mobile verification codes; do not use verification codes on computers. Add hacker IPs to the blacklist. Disconnect office computers from the internet, redo the system. Check bastion host logs, check server backdoors. Conduct deep security checks on all servers. Prohibit outgoing connections, block hacker remote control actions. Change all passwords, keys, and tokens. Limit access to the same interface to 10 times per minute. Deploy a cloud WAF that only enables OWASP defense functions. Add verification codes for human recognition to interfaces. Add a cloud lock, implement frequency limiting, iptables can also work. Create a whitelist. Change to a complex password or use keys. Add hacker IPs to the blacklist. Prohibit outgoing connections, block hacker remote control actions. Do not allow direct access to servers to prevent forward shells; use proxies instead. Change passwords or keys. Conduct deep security checks. Identify the source of code leaks and the leak pathways; modify everything. Add hacker IPs to the blacklist. Ensure all office network IPs are not on the whitelist; use VPN for whitelisting. Machines not in use or after hours should be powered down, and systems should be redone. Change all passwords or add two-factor authentication; use mobile phones as verification codes. Implement rate limiting for IPs. Use CDN, such as Cloudflare, add https. Fix the vulnerabilities that allow Baidu snapshots to hijack servers. Use HTTPDNS to check front-end js code, oss, clear CDN cache, and front-end code servers.

1. Define dynamic link library checking methods (dynamic link libraries are in /usr/lib64): echo $PATH echo $LD_PRELOAD cat /etc/ld.so.preload ls /etc/ld.so.conf.d / cat /etc/ld.so.conf. Use the strace command to check system commands: strace -f -e trace=file /bin/ls.

2. Symptoms of trojan dynamic link libraries: results from using ordinary commands differ from those using busybox; some commands show modified times when viewed with stat; ldd ELF filename (if the library is not present in a normal system, it is a trojan).

3. Repair methods: ./busybox lsattr -ia /etc/ld.so.preload ./busybox chattr -ia /etc/ld.so.preload ./busybox rm -rf/etc/ld.so.preload /lib/cub3.so.

4. Emergency Wrap-Up Work

Remove trojans and backdoors, scan for weak passwords, clean up preloaded libraries, trace back, write security incident reports, patch vulnerabilities, conduct penetration testing. Use NIDS to find hosts with abnormal traffic and locate intrusion points. Implement security hardening, clear alarms, and eliminate noise. Review bastion host logs to see if the operations were conducted by insiders. Use logs to find web vulnerability penetration interfaces. Analyze hacker penetration thinking, think from their perspective. Check if the internal network has been penetrated. Finally, the front end should implement signing, encrypt js, and encrypt request parameters; back-end verification is sufficient.

1. Permanent Solutions that do not affect other projects or networks. Application migration, patching, data center IPs should not be directly exposed to the outside, an additional layer of proxy can be added. Internal and external network firewall strategies and install Anqi. Identify critical machines and focus on defense.

2. Provide security recommendations based on the principle of least privilege. Ensure unified management entrance (bastion host), conduct penetration testing, prohibit connections to office computers and servers during non-working hours. For machines that must access the external network, a proxy or temporarily disabling the firewall will suffice. All machines should be quickly migratable. Continuously monitor and detect similar alarms. Store logs on servers to prevent hackers from clearing traces. Provide security awareness training.

Original text: https://github.com/dahailinux/Security-response-process

If there is any infringement, please contact us privately to delete the article.