Algorithm Optimization for Enhancing Intrusion Detection Systems in Gigabit Networks

1. Intrusion Detection Definition

1. Intrusion: Refers to a series of actions that attempt to compromise the confidentiality, integrity, and availability of information resources. This includes unauthorized access to an information system and/or unauthorized operations within an information system.
2. Intrusion Detection: It involves collecting information from several critical nodes within a computer network system and analyzing this information to monitor for any security policy violations or intrusions. It is the process of identifying and responding to malicious acts aimed at computing and network resources.
3. Intrusion Detection System (IDS): An Intrusion Detection System monitors the state and activities of protected systems using anomaly detection or misuse detection to identify unauthorized or malicious system and network activities. It provides an effective means to prevent intrusions and is a critical component of a comprehensive network security framework. The combination of intrusion detection software and hardware serves as a reasonable supplement to firewalls, forming the second security gateway after firewalls.
4. Contents of Intrusion Detection:

2. Typical IDS Technologies

• Origins and Development of IDS: Audit Technology: The process of generating, recording, and examining system event records in chronological order, often recorded in the form of audit logs.
• Goals of Auditing
  1. Determine and maintain accountability for each individual’s activity within the system
  2. Reconstruct events
  3. Assess losses
  4. Monitor problematic areas of the system
  5. Provide effective disaster recovery
  6. Prevent improper use of the system
• Intrusion Detection Process: Information collection, information analysis, result processing

1. Information Collection
2. Information Analysis

• Pattern Matching: This involves comparing the gathered information against a database of known network intrusion and system misuse patterns to identify behaviors that violate security policies.
• Statistical Analysis: Initially, a statistical description of system objects (such as users, files, directories, and devices) is created to capture certain measurement attributes under normal usage (e.g., access frequency, operation failure rate, and delays). The average values of these attributes are then used to compare against the behavior of networks and systems. Any observed values outside of the normal range may indicate an intrusion.
• Integrity Analysis is often used for post-analysis: It primarily focuses on whether a specific file or object has been altered. This often includes the content and attributes of files and directories and is particularly effective in discovering changed applications and trojan-installed programs.

1. Event Response (Result Processing):

3. Classification of Intrusion Detection Systems

• Host-based Intrusion Detection Systems, Network-based Intrusion Detection Systems, Distributed Intrusion Detection Systems

1. Host-based Intrusion Detection Systems (Host-based IDS, HIDS)

• Host-based Intrusion Detection Systems are installed on protected hosts and analyze and inspect the host’s real-time network connections and system audit logs. When suspicious behaviors or security violations are detected, the system alerts the administrator to take action. These protected hosts can include critical ones like web servers, mail servers, DNS servers, etc.
• Host Data Sources: Operating system event logs, application logs, system logs, relational databases, web servers.
• Detection Contents: System calls, port calls, system logs, security audits, application logs.
• Advantages of HIDS: High detection accuracy. HIDS is targeted at user and system activities, making it more suitable for detecting internal user attacks or unauthorized activities. Unaffected by encryption and switching devices. HIDS focuses solely on events occurring on the host itself, disregarding network events beyond the host, so its detection performance is not affected by data encryption, tunnels, and switching devices. Unaffected by network traffic. HIDS does not capture network packets and thus does not lose monitoring of system behavior due to increased network traffic, meaning its detection performance is independent of network traffic.
• Disadvantages of HIDS: HIDS is installed on the host that needs protection, inevitably consuming the host’s system resources, resulting in an additional load that reduces the efficiency of the application system. HIDS relies entirely on the operating system’s inherent auditing mechanism, necessitating tight integration with the operating system, which leads to poor platform portability. The robustness of HIDS is also limited by the security of the host’s operating system. HIDS can only detect attacks directed at the local machine and cannot detect network protocol-based attacks.

1. Network-based Intrusion Detection Systems (NIDS): Installed within the segment that needs protection, it monitors various data packets transmitted in real-time within the segment and analyzes and detects these packets. If intrusion behaviors or suspicious events are identified, the intrusion detection system will alert or even disconnect the network connection.

• Network Monitoring: On a shared network, the ability to listen to all traffic is a double-edged sword. Administrators can use it to monitor network traffic conditions. Developers of network applications can monitor the network conditions of the program. Hackers can use it to gather network intelligence.
• Advantages of NIDS: Fast detection and response speed. NIDS can identify attacks and suspicious intentions before a successful intrusion, enabling rapid response to halt the attack process before the target suffers damage. Wide intrusion monitoring range. As each network sensor can capture all data packets within a shared segment, a single network sensor can protect an entire segment. Therefore, installing network sensors on key network paths allows monitoring of the entire network communication. Reliable intrusion forensics. NIDS collects intrusion evidence by capturing data packets, which attackers cannot transfer. Ability to detect protocol vulnerability attacks. Many attack programs are written based on network protocol vulnerabilities, such as SYN flood, Smurf attack, and teardrop attack, which can only be identified by examining packet headers or payloads.

1. Distributed Intrusion Detection Systems (DIDS): The complexity and enlargement of network system structures cause weaknesses or vulnerabilities to be dispersed across hosts in the network, which attackers might exploit to attack the network. Relying solely on a host or network intrusion detection system makes it challenging to identify intrusion behaviors. Intrusion behaviors are no longer singular but exhibit collaborative intrusion characteristics, such as distributed denial-of-service attacks. The decentralization of data sources crucial to intrusion detection makes collecting raw detection data relatively challenging.

• The goal of Distributed Intrusion Detection Systems (DIDS) is to detect both network and host intrusion behaviors.
• Location of the Detector:
• Problems Facing Network-based Technologies: In network environments utilizing switching technology, the switching mechanism prevents network packets from being broadcast within the subnet at will, broadcasting only within specific virtual networks (VLANs). As a result, hosts performing network monitoring can only extract data within their VLAN, greatly reducing the monitoring range, and limiting their monitoring capability.

4. Intrusion Detection Methods

1. Misuse Detection

• Misuse Detection, also known as signature-based detection, involves characterizing descriptions of intrusion behaviors to establish patterns of certain or various intrusion behavioral features. If current behaviors align with any intrusion pattern, it indicates that intrusion has occurred.
• Characteristics of Misuse Detection:

1. Anomaly Detection

• Basic Idea: Any individual’s normal behavior follows a certain pattern, and this pattern can be summarized by analyzing log information generated by these behaviors (assuming the logging information is sufficiently comprehensive). In contrast, intrusion and misuse behaviors generally differ significantly from normal behaviors. By identifying these differences, intrusions can be detected.
• Main Methods: Establish a rule set for normal behavior, known as a normal behavior pattern, also referred to as a normal profile or “user profile.” When user activities deviate significantly from the normal profile, they are considered abnormal or intrusive behaviors. This approach can detect illegal intrusion behaviors, even those carried out using unknown attack methods. Moreover, non-intrusive abnormal user behaviors (abuse of one’s access rights) can also be detected.
• Characteristics of Anomaly Detection:
• Methods Used in Anomaly Detection:

1. Statistical Anomaly Detection
2. Feature Selection-based Anomaly Detection
3. Bayesian Inference-based Anomaly Detection
4. Bayesian Network-based Anomaly Detection
5. Pattern Prediction-based Anomaly Detection
6. Neural Network-based Anomaly Detection
7. Bayesian Clustering Anomaly Detection
8. Machine Learning-based Anomaly Detection
9. Data Mining-based Anomaly Detection

• Comparison of the Two Approaches
• Development Directions for Intrusion Detection:

1. Industry:

The main research focus is on how to enhance the comprehensive performance and processing speed of intrusion detection systems through algorithm optimization to meet the demands of gigabit networks.

1. Academia:

The focus is on introducing various intelligent computing methods, guiding intrusion detection technology towards intelligent development. Technologies such as artificial neural networks, artificial immune systems, and data mining are being explored.

• Limitations of Intrusion Detection Systems:

1. The conflict between false positives and false negatives
2. The contradiction between privacy and security
3. The conflict between passive analysis and active discovery
4. The contradiction between massive information and analysis costs
5. The conflict between functionality and manageability
6. The contradiction between a single product and complex network applications

5. Network Intrusion Detection System Products

• Snort is the most popular free NIDS. Snort is an IDS based on misuse/anomaly detection and employs rule definitions to inspect problematic packets in the network. Snort consists of several parts: packet sniffer, preprocessor, detection engine, and alert output module.
• RealSecure: In 1996, RealSecure was initially developed as a traditional sensor-based network intrusion detection system. By 1998, it became a hybrid intrusion detection system. It strives to provide a hybrid performance of OS logs and network packet capabilities, designed to be placed both below and above the IP layer of the protocol stack, offering multiple response approaches.
• Network ICE