Memory Scanning for Malicious Activities: Comprehensive Investigation and Countermeasures

Use the commands stat /tmp/.X291-unix/.rsync/a/a and file /tmp/.X291-unix/.rsync/a/a to confirm the existence of the malicious file /tmp/.X291-unix/.rsync/a/a mentioned in the alert.

If the alert file cannot be located at this step, it might have been self-deleted by malicious software. In that case, investigate other files based on the alert time. If still unsuccessful, memory scanning might be required.

Given that the alert’s malicious file is a shell script, use the command cat /tmp/.X291-unix/.rsync/a/a to check its contents. Behaviors like clearing scheduled tasks, connecting to malicious IPs, and detecting system type indicate this shell script is malicious.

Because the alert file’s path is unusual, use the command find /tmp/.X291-unix/ -type f -exec ls -lctr --full-time {} \+ 2>/dev/null to view other files in the path. Many malicious files are found in /tmp/.X291-unix/, all originating from the earliest extracted /tmp/.X291-unix/dota3.tar.gz. All these malicious files should be backed up and then deleted.

02. Process

The command lsof /tmp/.X291-unix/.rsync/a/a showed no processes related to the alert file.

03. Network

Since no processes related to the alert file were found, there are no network connections associated with it either.

1.1.2. Time File Examination

Use the command find / -newerct '2024-01-27 19:21:00' ! -newerct '2024-01-27 21:21:00' ! -path '/proc/*' ! -path '/sys/*' ! -path '/run/*' -type f -exec ls -lcr --full-time {} \+ 2>/dev/null (Note: the ls command is not using the -t parameter to sort by filename) to examine files created around the malicious program’s drop time. Four groups of suspicious files were found.

01. /var/log/*
Use the command less filename to check five log files in /var/log/. The btmp file is binary, so use lastb -f /var/log/btmp-20240127 to view it. No anomalies were found in any files, so they can be ignored.

02. /tmp/.X291-unix/.rsync/*
Files in this directory were analyzed in 1.1.1. They all need to be backed up and deleted later.

03. /root/.configrc5/*
Files in this directory resemble those in /tmp/.X291-unix/.rsync/*, indicating they were copied. They all need to be backed up and deleted later.

04. /var/tmp/.systemcache436621
Using the command cat /var/tmp/.systemcache436621 shows the file content as only a 1, with no malicious content. It might just be for signal transmission, and further analysis will determine whether it should be deleted.

05. /var/spool/cron/root
Attackers created many scheduled tasks involving malicious files in `/root/.configrc5/` and `/tmp/.X291-unix/.rsync/`. These backdoor scheduled tasks need to be deleted, and both directories’ malicious files must be backed up and deleted later.

06. /usr/local/qcloud/YunJing/QuaraV2/package_d01cee3bf47bbe9cd18505e4189baa2e.zip
Based on the path, this is an installation package for Tencent Cloud’s host security product YunJing. Unzipping it reveals a previously extracted program named `YDQuaraV2`. Sandbox analysis shows it isn’t malicious and can be ignored.

07. /tmp/up.txt
Using the cat command to view the file reveals SSH service account passwords. What’s the point of storing my account passwords? Are they planning to undo their mischief later? This file must be backed up and deleted later.

08. /etc/shadow- and /etc/shadow
Using the cat command to view `/etc/shadow-` reveals the password, which CMD5[1] decodes as 123456. It appears to be the original shadow file. It’s puzzling why `/etc/shadow-` would store old SSH service passwords rather than `/tmp/up.txt`. Though not malicious, it should be backed up and deleted later, as it contains a weak password.

Using the cat command to view /etc/shadow and then CMD5 to decode it failed, indicating the attacker set a strong password. Great security awareness—a solid “know thy enemy” mentality. The root user’s password should be changed later.

09. /etc/hosts.deny
Using the cat command shows the attacker cleared the `/etc/hosts.deny` file, likely to prevent UEBA rules from detecting successful brute-force login attempts and automatically blocking the attacker’s IP. This constant clearing of /etc/hosts.deny poses a risk; if UEBA is slow, it could complete attack detection and blocking 10 minutes later. No further action required.

1.1.3. Sensitive Directory Examination

Using the command find /tmp/ -type f -exec ls -lctr --full-time {} \+ 2>/dev/null to examine the temporary directory, no new suspicious files were found.

Using find $HOME ! -type d -exec ls -lctr --full-time {} \+ 2>/dev/null to check the home directory also revealed no new suspicious files. However, it was discovered that /root/.ssh/authorized_keys was modified at an unexpected time. The file contains an attacker’s SSH public key, which needs to be backed up and removed later.

1.1.4. Privileged File Examination

The command find / -perm -u=s -type f -exec ls -lctr --full-time {} \+ 2>/dev/null was used to examine privileged files. No suspicious files were found.

1.2. Network Examination

Using the command netstat -tunlap revealed many connections to the internet by the 14720/httpd process, likely performing a vulnerability scan. This process must be terminated later.

Furthermore, using the command netstat -tunlap | grep -v "14720/httpd" | grep -v "WAIT" revealed a suspicious process 19080/./kswapd0. Further analysis is required, and access to 179.43.139.84 should be blocked, as threat intelligence warns it is a malicious entity.

1.3. Process Examination

1.3.1. Network Process Examination

Using the command pstree -asp 14720 | head revealed all parent and child processes of the suspicious process 14720/httpd, totaling 520. All of these need to be terminated later. Hence, it’s vital not to kill the anomalous process detected in 1.2. Network Examination immediately; it would only kill one malicious process while missing the other 520.

The command ls -l /proc//cwd /proc//exe was used to view the startup directory and program of the malicious process, revealing all are within /tmp/.X291-unix/, previously discovered in 1.1.1. Alert File Examination. All these malicious programs must be backed up and deleted.

Utilizing the command pstree -asp 19080 revealed all parent and child processes of the suspicious 19080/./kswapd0. These must be terminated later.

The command ls -l /proc//cwd /proc//exe inspected the startup directory and program of the malicious process, revealing these were found in 1.1.2. Time File Examination‘s /root/.configrc5/ directory. These malicious programs must be backed up and deleted.

1.2.2. Examination of All Processes

The command pstree -as was used to examine all processes, but no new suspicious processes were found through process names and startup commands.

1.2.3. Hidden Process Examination

The commands ps -ef | awk '{print}' | sort | uniq > 1, ps -ef | awk '{print}' | sort | uniq > 2, and diff 1 2 were employed to uncover hidden processes. Aside from the known malicious process 19080/./kswapd, no other detections were made.

1.2.4. Process Resource Examination

Using the command top to examine resource consumption by all processes revealed no new insights beyond the known malicious processes 19080 and 14720 pushing CPU usage to nearly 200%.

2. Backdoor Examination

2.1. Account Backdoor Examination

2.1.1. SSH Accounts

The command cat /etc/passwd | grep -v 'nologin\|false' was used to examine SSH-accessible accounts, revealing only root and lighthouse accounts had /bin/bash enabled, with no anomalies found.

The command cat /etc/passwd | awk -F: '3==0 {print 1}' checked for UID 0 superuser accounts, finding only the root account without any anomalies.

The command cat /etc/shadow | awk -F: 'length(2)==0 {print 1}' checked SSH accounts for null passwords, finding no anomalies.

2.1.2. SSH Keys

The privileged user’s /root/.ssh/authorized_keys file, analyzed in 1.1.3. Sensitive Directory Examination, showed no SSH key files for ordinary users.

2.2. Conditional Backdoor Examination

2.2.1. Scheduled Tasks

The command find /var/spool/cron/ -type f -exec ls -lctr --full-time {} \+ 2>/dev/null was used to examine scheduled tasks, showing the /var/spool/cron/root file was tampered with by attackers on 2024-01-27 at 20:41:21, analyzed in 1.1.2.5.

The command find /etc/*cron* -type f -exec ls -lctr --full-time {} \+ 2>/dev/null was used to examine scheduled tasks, with no anomalies found in suspicious tasks viewed with the cat command.

2.2.2. Startup Items

The command find /etc/rc.d/ -type f -exec ls -lctr --full-time {} \+ 2>/dev/null was employed to examine startup items, with no anomalies detected in suspicious items viewed with the cat command.

2.2.3. Auto-start Services

The commands chkconfig --list and service --status-all were used to examine auto-start services, with no anomalies found.

2.2.4. Command Aliases

Using the command alias to inspect command aliases, no anomalies were detected.

The command find / -name *bashrc* -type f -exec ls -lctr --full-time {} \+ 2>/dev/null was used to examine configuration files that might configure command aliases, with no anomalies found.

3. Traceback Examination

3.1. Log Analysis

3.1.1. System Logs

01. Security Logs (secure)

The command cat /var/log/secure* | grep Accepted | awk '{print $11}' | sort | uniq -c | sort -nr revealed five unknown malicious IPs that successfully logged in via SSH each once.

Threat intelligence confirmed 43.139.33.6 as a malicious IP from Guangzhou. The command cat /var/log/secure* | grep 43.139.33.6 | grep Failed | wc -l showed five failed login attempts, and cat /var/log/secure* | grep 43.139.33.6 | grep Accepted | wc -l showed one successful login, indicating a brute-force success.

The command cat /var/log/secure* | grep 43.139.33.6 was used to check login time, revealing success at 2024-01-27 20:20:27, coinciding with this mining incident’s attack time. Tencent Cloud’s alert, however, lacked warning of this malicious IP’s successful password brute-force and login, showing Tencent Cloud’s leniency towards domestic malicious IPs.

Threat intelligence confirmed 213.57.175.148 as a malicious IP from Beijing. The command cat /var/log/secure* | grep 213.57.175.148 | grep Failed | wc -l showed four failed login attempts, and cat /var/log/secure* | grep 213.57.175.148 | grep Accepted | wc -l showed one successful login, indicating a brute-force success.

The command cat /var/log/secure* | grep 213.57.175.148 was used to check login time, showing the brute-force attempt started at 2024-01-27 19:53:02 and succeeded in seven seconds, unrelated to this mining incident’s attack time.

Despite the successful brute-force, no automated scripts were executed, possibly intending a manual lateral move within the network for ransomware deployment. Regardless, they should have changed the SSH weak password first to avoid the server being hijacked by this mining virus’s attacker.

02. Lastlog Series

The command lastlog checked users’ last login information, and the command w checked users currently logged in. Neither revealed anomalies.

03. Command Log (history)

Using the history command (/root/.bash_history) to inspect command history, no attacker-used commands were found.

3.1.2. Middleware Logs

No middleware services are running, hence no middleware log analysis.

3.1.3. Database Logs

No database services are running, hence no database log analysis.

3.1.4. Security Device Logs

No security devices are deployed, hence no security product log analysis.

3.2. Traffic Analysis

No traffic collection products are deployed, hence no traffic analysis.

3.3. Memory Analysis

Given the current known situation, memory analysis is deemed unnecessary.

3.4. Traceback Conclusion

01. During 2024-01-27 20:20:27

The attacker from Guangzhou with IP address 43.139.33.6 successfully brute-forced the SSH root account.

02. Between 2024-01-27 20:20:32 to 20:41:22

Five seconds later, the attacker uploaded and executed malicious programs /tmp/.X291-unix/* and /root/.configrc5/*, while modifying backdoor configurations like /etc/shadow, /root/.ssh/authorized_keys, and /var/spool/cron/root, a process lasting 21 minutes.

Notably, the /root/.ssh/authorized_keys file’s modification did not occur within this timeframe.

03. During 2024-01-27 20:21:02

The malicious program /tmp/.X291-unix/.rsync/a/a was uploaded by the attacker at 24-01-27 20:21:00. Tencent Host Security completed threat detection and alert notification just two seconds later at 20:21:02, demonstrating impressive capability.

Intriguingly, although 40 files were released during this attack, only one was detected.

4. Post-Incident Tasks

4.1. Terminate Malicious Processes

No.	Task	Reason	Method
1	Terminate malicious process 14720/httpd	Excessive internet connections, likely due to vulnerability scanning.	kill -9 14720…
2	Terminate malicious process 19080/./kswapd0	Connection to Swiss malicious IP 179.43.139.84. Function yet to be analyzed, likely a remote control backdoor.	Kill -9 19089…

4.2. Backup and Delete Malicious Programs

No.	Task	Reason	Method
1	Backup and delete /tmp/.X291-unix/*	Extensive internet connections, likely due to vulnerability scanning.	scp -r [email protected]:/tmp/.X291-unix/ ./ rm -rf /tmp/.X291-unix/
2	Backup and delete /root/.configrc5/*	Connection to Swiss malicious IP 179.43.139.84. Function yet to be analyzed, likely a remote control backdoor.	As above.
3	Backup and delete /var/tmp/.systemcache436621	Potential use by malicious program for signal transmission.	As above.
4	Backup and delete /tmp/up.txt	SSH service account passwords.	As above.

4.3. Fix Vulnerabilities and Backdoors

No.	Task	Reason	Method
1	Change root user password	Weak password vulnerability allowed attacker server access	passwd root
2	Backup and delete /root/.ssh/authorized_keys	Attacker’s SSH backdoor key	scp [email protected]:/root/.ssh/authorized_keys ./ rm -rf /root/.ssh/authorized_keys
3	Backup and modify /var/spool/cron/root	Attacker’s scheduled task backdoor	1. Open the file: vim /var/spool/cron/root 2. Navigate: Use arrow keys to reach malicious code line 3. Delete code: Enter dd to remove the malicious code 4. Exit file: :wq

4.4. Attack Backtrace and Protection

No.	Task	Reason	Method
1	Defense: Block access requests to malicious IP 179.43.139.84 on the border firewall.	179.43.139.84 is likely a remote control server.	Refer to the border firewall’s operation manual.
2	Traceback: Check traffic-side security devices for related requests in the last 7 days. If found, investigate if the source device is infected with a mining virus.	As above.	Refer to traffic-side security devices’ operation manuals.

Sax2 Intrusion Prevention System