Protecting IT Systems from Holiday DOS Attacks: A Comprehensive Guide

Website Hit by DOS Attack

I. Incident Background

The long holidays are a brief rest period for IT staff, but IT systems cannot afford to stop even for a moment. The more it’s a holiday, the more likely major issues arise; what follows is a case of a DOS attack.

Just after the Spring Festival holiday, Xiao Li’s company’s web server experienced a malfunction. At 1 PM, after lunch, Xiao Li habitually checked the web server. The traffic monitoring system showed a descending red line, and an email alert was received at the same time, indicating that there was an issue with the server.

Based on the above problem, Xiao Li immediately started checking the web server logs, attempting to find some clues related to the interruption. During the investigation, the department manager informed Xiao Li that he had received a customer complaint call stating they could not access their website.

Nothing suspicious was found in the web server’s log files, so Xiao Li next examined the firewall logs and router logs. He printed the records of the problematic server time, filtering out normal traffic and preserving suspicious records. Table 1 shows the printed results.

Table 1 Firewall Log Statistics

Source IP Address	Destination IP Address	Source Port	Destination Port	Protocol
172.16.45.2	192.168.0.175	7843	7	17
10.18.18.18	192.168.0.175	19	7	17
10.168.45.3	192.168.0.175	34511	7	17
10.18.18.18	192.168.0.175	19	7	17
192.168.89.111	192.168.0.175	1783	7	17
10.18.18.18	192.168.0.175	19	7	17
10.231.76.8	192.168.0.175	29589	7	17
192.168.15.12	192.168.0.175	17330	7	17
10.18.18.18	192.168.0.175	19	7	17
172.16.43.131	192.168.0.175	8935	7	17
10.23.67.9	192.168.0.175	22387	7	17
10.18.18.18	192.168.0.175	19	7	17
192.168.57.2	192.168.0.175	6588	7	17
172.16.87.11		10.18.18.18	19	7	17

He did the same analysis on the router logs and printed the seemingly abnormal records. Table 5-1 shows the normalized router log information during the attack on the website.

To obtain more information, Xiao Li continued to inspect the router’s NetFlow aggregated statistics, details as follows:

To have a reference benchmark, he also printed cached data saved from a few weeks before the web server began experiencing problems (these are normal state data). The normal routing logs are as follows:

Under the title “IP packet size distribution,” two lines show the percentage distribution of packets by size range. Here, it is indicated that only 2% of packets are between 33 and 64 bytes in size.

Notice the sharp decline in website traffic. Clearly, nobody could access the web server during this time. Xiao Li began investigating what happened and how to quickly remedy the situation.

II. Q&A

1. What exactly is happening to Xiao Li’s web server? What could be the possible types of attacks?

2. How can Xiao Li trace the attacker if the addresses are not spoofed?

3. How can he track an attacker if the addresses are spoofed?

III. Incident Reasoning

What kind of attack did Xiao Li’s web server suffer from? This attack was carried out by continually sending UDP packets through the echo port (echo port number 7). The attack seemed to originate from two places, possibly two attackers using different tools. In any case, the overload of data streams would cripple the web server. However, the source of the attack address is uncertain; it’s difficult to determine whether the attack source is distributed or a single real address masquerading as many different fake IP addresses. If the source IP address isn’t spoofed, consult ARIN, the American Internet Numbers Registry, to find out which network the invading IP address belongs to by checking its “Whois” database. Then, contacting that network’s administrator could provide further information, although this is unlikely for DOS attacks.

If the source address is spoofed, tracing the attacker becomes much more complex. If using a Cisco router, it would require querying the NetFlow cache. However, to trace the spoofed address, each router’s NetFlow cache must be inspected to determine which interface the traffic entered through and then tracked back through these router interfaces to find the IP address source. Nevertheless, this is highly challenging, as numerous routers may be between the web server and the attacker’s originating PC, often involving different organizations. Moreover, this analysis must be performed during the attack. Without judicial intervention, tracing the source is extremely difficult.

After analysis, correlating information from the firewall logs and router logs revealed some interesting similarities, marked in bold black in Table 5-1. The target of the attack was clearly the web server (192.168.0.175, port UDP 7). This strongly resembles a denial of service attack (though it cannot be confirmed since the source IP addresses are randomly distributed). The addresses appear random, except for one constant source address whose source port number also remained unchanged. This is interesting. He then focused on the router logs again.

He discovered that during the attack, the router logs indicated a flood of 64-byte packets, while the web server logs showed no issues. He also observed a significant number of “UDP-other” packets logged on the router system at the time of the event, while the web server logs remained wholly normal. This occurrence is consistent with a hypothesis of a UDP-based denial of service attack.

It could be hypothesized that the attacker was indeed filling the web server’s echo (echo 7) port with a torrent of small UDP packets, so Xiao Li and his team’s next task was to stop this attack. First, Xiao Li blocked the attack on the router. He quickly set up a filtering rule for the router. Given the randomness of the source address origins, it seemed challenging to prevent the attack by restricting certain addresses or a specific range of addresses, so they decided to block all UDP packets destined for 192.168.0.175. This action would result in the server losing some functionality, like DNS, but at least allow the web server to operate normally.

The router’s initial temporary DOS access control list (ACL) was as follows:

access-list 121 remark Temporary block DoS attack on web server 192.168.0.175

access-list 105 deny udp any host 192.168.0.175

access-list 105 permit ip any any

This approach alleviated the burden on the web server, yet the attack still affected the web, reducing network performance to some extent. Therefore, the next step was to contact the upstream bandwidth provider to temporarily restrict all UDP inbound traffic on Xiao Li’s site port 7, significantly decreasing the bandwidth on the network to the server.

IV. Countermeasures

There is no panacea for preventing and mitigating these bandwidth-related DOS attacks. Essentially, it’s a “bigger pipe beats smaller pipe” attack. The attacker, sometimes commanding immense bandwidth, can overwhelm a network that lacks sufficient bandwidth. In this situation, prevention and mitigation must complement each other.

There are several measures to make attacks more challenging or reduce their impact if they occur, as follows:

Ingress filtering at network entry: Network service providers should set up ingress filtering on their downstream networks to prevent fake packets from entering the network. This will stop the attacker from forging IP addresses, making traceability easier. Filtering unnecessary traffic with network traffic filtering software is always a good idea. This can prevent DOS attacks, but for effectiveness, these filters should be placed as far upstream in the network as possible.

Network traffic rate limiting: Some routers have a maximum traffic rate limitation. These limits support bandwidth policies and allow a given type of network traffic to match a limited bandwidth. This measure can preemptively mitigate ongoing attacks.

Intrusion detection systems and host monitoring tools: IDS can alert network administrators to attack timings and the tools attackers use, assisting in stopping the attack. Host monitoring tools can alert administrators if DOS tools appear within the system

Single-packet RPF (Reverse Path Forwarding): This Cisco Express Forwarding (CEF) router feature checks whether the incoming interface has a routing entry for the packet’s source IP address. If the incoming interface’s routing doesn’t match, the router drops that packet. The allure of dropping RPF packets is that it blocks all attacks with spoofed source IP addresses.

1) Detecting DOS Attacks

With a combination of host monitoring systems and IDS systems, issues can be quickly detected, such as using the EtherApe tool (an open-source tool for monitoring connections). Similar results can be achieved with Sniffer Pro and Colliery network analysis tools. Sniffer can display the network connection status in real-time. If a DOS attack occurs, the dense internal lines and IP addresses can initially determine the type of attack, then OSSIM system traffic monitoring software like Ntop and IDS systems can be applied for detailed judgment. These will be elaborated in the book “Unix/Linux Network Log Analysis and Traffic Monitoring.” The quickest method still involves using the command line. Enter the following command:

# netstat -an|grep SYN_RECV|wc –l

By checking the result, it’s possible to detect a large number of TCP synchronization packets with very few successfully established TCP connections. Based on the TCP three-way handshake principle, this is certainly not normal, indicating network issues that need further investigation. If the number is very high, say over a thousand, it is very likely an attack, as illustrated in Figure 1.

Figure 1 Ossim Detecting DOS Attack

In Figure 1, the OSSIM system’s Snort detected a DOS attack and graphically displayed numerous alerts. For instance, when a website is subject to a DOS attack, TCP connections appear as follows:

By counting the number of “SYN_RECV” statuses, the command is as follows:

#netstat –na |grep SYN_RECV |wc –l

1989

This large number, combined with the diagram above, indicates a DOS attack on the website.

A tip: The following shell command can also display which IP has the most connections.

#netstat -nta |awk ‘{print $5}’ |cut –d:f1 |sort|uniq –c |sort –n

1 192.168.150.10

2 192.168.150.20

… …

1987 192.168.150.200

This command provides more detailed information. With a count of 1989 making nearly two thousand entries, it clearly indicates a DOS attack. Using the Wireshark tool to decode data packets can reveal more issues. The current communication entirely uses the TCP protocol, and viewing the TCP flags shows all packets are SYN set to 1, which are TCP synchronization request packets targeting a specific IP address. Up to this point, the earlier assumptions are validated: this host suffered from a DOS attack, specifically a SYN Flood attack.

V. Troubleshooting

1. Xiao Li’s server was subjected to a DOS attack, realized by continually sending small UDP packets to port 7. The attack appeared to originate from two places, potentially separate attackers using different tools. The major influx of data quickly overwhelmed the web server. The challenge lies in determining whether the attack source addresses are distributed or a single address pretending to be numerous different IP addresses is not straightforward.

2. If the address isn’t spoofed, Xiao Li can search ARIN’s Whois database to determine which network the intruding IP address belongs to.

3. If the IP address is spoofed, tracking becomes complicated, requiring consultation of each router’s NetFlow data to pinpoint where the traffic entered and then tracing back through these routers one interface hop at a time until finding the originating IP address source. This involves multiple AS (autonomous systems), and if tracing domestically involves many providers and law enforcement, extending the workload and time significantly, and international tracing becomes even more complex. The hardest part is conducting this analysis must occur during an attack to ensure accuracy. Once the attack concludes, the only option is to consult the logging system.

From the actual case above, it’s evident many DoS attacks are difficult to counter because the requests from the disruptive host are entirely legal and standard, just vastly numerous. Initially, router ACLs can be used to block ICMP echo requests.

Router(config)#ip tcp intercept list 101

Router(config)#ip tcp intercept max-incomplete high 3500

Router(config)#ip tcp intercept max-incomplete low 3000

Router(config)#ip tcp intercept one-minute high 2500

Router(config)#ip tcp intercept one-minute low 2000

Router(config)#access-list 101 permit any any

If using Context-Based Access Control (CBAC), configuring timeout and threshold settings can combat SYN floods and UDP garbage floods.

Router(config)# ip inspect tcp synwait-time 20

Router(config)# ip inspect tcp idle-time 60

Router(config)# ip inspect udp idle-time 20

Router(config)# ip inspect max-incomplete high 400

Router(config)# ip inspect max-incomplete low 300

Router(config)# ip inspect one-minute high 600

Router(config)# ip inspect one-minute low 500

Router(config)# ip inspect tcp max-incomplete host 300 block-time 0

Warning: It’s recommended not to use both TCP intercept and CBAC defense features simultaneously because this could overload the router.

Enabling Cisco Express Forwarding (CEF) can assist the router in defending against floods with randomly set source addresses. Scheduler settings can be adjusted to avoid complete router CPU overload during a flood:

Router(config)#scheduler allocate 3000 1000

After configuration, IOS allocates 3 seconds to handle network interface interrupt requests and 1 second to perform other tasks. For earlier systems, use the command scheduler interval.

Another approach is using DOS prevention scripts with Iptables:

#!/bin/bash

netstat -an|grep SYN_RECV|awk ‘{print $5}’|awk -F: ‘{print $1}’|sort|uniq -c|sort –rn|awk ‘{if ($1 >1) print }

for i in $(cat /tmp/dropip)

do

/sbin/iptables -A INPUT -s $i -j DROP

echo “$i killed at `date`” >>/var/log/ddos

done

This script compiles statistics on IPs with 5 or more SYN_RECV counts and writes them to Iptables’ INPUT chain setting to drop.

VI. Case Summary

No matter the intent behind larger-scale attacks or other DOS/DDoS objectives, these must be taken seriously. Preventing such attacks primarily involves promptly applying patches from vendors, closing vulnerable services, or using access control lists to limit access. Standard DOS attacks, especially DDoS attacks, are more challenging to prevent. If the entire bandwidth is consumed by a ping flood, our options are very limited. For DOS attacks, the first step is to analyze the attack method, whether it’s ICMP Flood, UDP Flood, or SYN Flood traffic attacks, or if it’s a TCP Flood, CC, or a similar approach, then seeking relatively effective countermeasures. For such attacks, the following approaches can be used:

1) Utilizing ”honeypots” for protection, enhancing first impressions analysis and response to attack tools and malicious samples. Deployment of large-scale honeypot devices to track botnet dynamics and capture malicious code. Deploy website monitoring devices to reinforce web page tampering, access redirection mechanisms, and domain name resolution monitoring, severing primary infection paths of malicious code. Utilize automated analysis devices with sandbox technology and various unpacking technology to enhance research of new malicious code and improve research timeliness.

2) Using the Apache DOS defense strategies provided by the Ossim system can serve a monitoring role.

3) Employing cloud computing and virtualization and other new technology platforms can improve the detection and defense efficacy against new emerging attacks, particularly application-layer and low-rate attacks. Scholars abroad are already utilizing the Hadoop platform to research HTTP Get Flood detection algorithms.

4) Employing IP reputation mechanisms: Integrating reputation mechanisms in each phase of information security defense to improve efficiency and accuracy. For example, offering safety reputation evaluations of applications and files to guide user downloads, and generating automatic protection strategies through authoritative IP reputation information. See Section 2.1 of “Unix/Linux Network Log Analysis and Traffic Monitoring” for details.

5) Adopting a passive strategy by purchasing larger bandwidth can also effectively mitigate DDoS attack damages.

6) Establishing distributed systems by deploying operations in multiple data centers can decentralize access across regional data centers. Deploying CDN and placing firewalls (e.g., Cisco, Juniper) at critical IDC node data centers means that even if attackers launch a DOS attack, only one data center might be affected, without impacting the entire business.

7) If the scale isn’t large, and data center conditions are average, consider using small anti-DDoS tools like DDoS Deflate. Its official website is http://deflate.medialayer.com. It’s a free script used to defend against and mitigate DDoS attacks by monitoring and tracking IPs creating numerous network connections through the system’s built-in netstat command. When exceeding a preset limit, it blocks or bans these IPs using APF or IPTABLES. However, this tool simply alleviates issues and cannot entirely prevent attacks.

Lastly, utilizing different vendors, different AS paths, and multiple internet connections supporting load balancing functions fall short of the requirements to combat conventional high-bandwidth DOS/DDoS floods. However, we can always use CAR (Committed Access Rate) or NBAR (Network-Based Application Recognition) to discard packets or limit the network flow speed initiating an attack, reducing the router’s CPU load, and minimizing host usage.

The content of this article is contributed by internet users. The views expressed are solely the author’s. This site only provides information storage space services, does not own all rights, and assumes no legal responsibility. If you find this site involved in infringement/violation, please email us to report. Once verified, the site will promptly delete it.

Publisher: Full Stack Developer Stack Leader, please indicate the source: https://javaforall.cn/187270.html Original link: https://javaforall.cn

ddosnetwork securitysecuritywebsitetcp/ip