Comprehensive Guide to Company Security: Mastering Intrusion Detection and Prevention

Preface

How can you determine if your company has been compromised? Is it that no one is trying to hack you, or is it due to insufficient awareness that you’re yet to discover an intrusion? In reality, intrusion detection is a formidable challenge that every large internet company must confront. The more valuable the company, the greater the threat of intrusion it faces, as even Internet pioneers like Yahoo have suffered data breaches at the end of their tenure (during acquisition). Security is no small matter, and the consequences of a successful intrusion into an internet company can be unimaginable.

From the perspective of “offensive and defensive confrontation,” this article will not address specific intrusion detection models, algorithms, and strategies, so those looking to directly copy “intrusion tactics” may be disappointed. However, we will share some operational insights for our colleagues’ feedback. If it aids future practitioners, that’s even better, and we welcome discussions and exchanges with everyone.

Definition of Intrusion

Typical intrusion scenarios:

Hackers remotely control the target’s laptop/mobile device/server/network equipment from afar through the network, arbitrarily accessing private data, or utilizing system features, including but not limited to using the mobile phone’s microphone to eavesdrop, using the camera for surveillance, mining cryptocurrency with the device’s computational power, and launching DDoS attacks through the network. Alternatively, cracking a service’s password to access sensitive data or control access systems/traffic lights. All of these constitute classic intrusion scenarios.

We can define intrusion as: unauthorized control and use of resources such as data and commands by hackers to achieve various goals. Broadly speaking, hackers exploiting SQL injection vulnerabilities to steal data, obtaining credentials at a domain registrar to tamper with DNS, or accessing social accounts to control virtual assets all fall under the category of intrusion.

Enterprise Intrusion Detection

Enterprise intrusion detection typically has a narrow focus, often referring specifically to hackers’ control over PCs, systems, servers, and networks (including corporate and production networks).

The most common method for hackers to control host assets like PCs and servers is by executing commands through Shell access, known as GetShell.

For example, through web service upload vulnerabilities to obtain a WebShell, or using RCE vulnerabilities to execute commands or code directly (RCE environments indirectly provide a Shell). Also, by implanting “Trojan backdoors,” and later using the SHELL feature of Trojans for remote control, which is quite typical.

Therefore, intrusion detection can focus on the GetShell action and malicious behaviors following GetShell success (as hackers often use the Shell to probe, steal, and launch lateral movement attacks on other internal targets, which differ from normal user behavior and can be important characteristics).

Some colleagues (including commercial products) prefer to report “external scans, attack probes, and attempt behaviors” before GetShell and call it “situational awareness,” indicating that someone is trying to attack. In my opinion, its practical value is limited. Many companies, including Meituan, are almost constantly under “unknown identity” attacks. Knowing someone is “attempting” to attack, if it does not lead to effective action or alert, just drains energy without providing substantial value.

Once you get used to “attacks” as the norm, you solve problems under this normalcy by using hardening strategies where applicable, and if some strategies cannot be implemented regularly and require overtime and temporary vigilance, those strategies tend to fade away shortly after. Whether or not we adopt such a strategy does not fundamentally change anything.

Similar to SQL injection, XSS, and other web attacks that do not directly involve GetShell, these are temporarily not considered within the narrow scope of “intrusion detection.” They can be categorized as “vulnerabilities,” “threat perception,” and further discussed separately. However, if GetShell actions are executed via SQL injection, XSS, or other entry points, we still focus on the GetShell key point, regardless of the vulnerability entry location.

Intrusion and Insider Threats

A scenario closely related to intrusion is “insider threats.” Intrusion is a method—GetShell is just a starting point, with hackers aiming for resource control and data theft afterwards. “Insiders,” on the other hand, have legitimate permissions to access sensitive assets legally but misuse these resources for unauthorized purposes, such as copying, transferring, leaking, and altering data for profit.

The behavior of insiders does not fall under “intrusion detection,” and it’s generally managed and audited from an internal risk control perspective, such as separation of duties and dual-person audits. Data Loss Prevention (DLP) products can also assist in this regard, though we will not elaborate here.

Sometimes, hackers know that Employee A has access to target assets, so they attack A, then use A’s permissions to steal data, still classified as “intrusion.” After all, A is not a malicious insider. If we can’t capture when hackers attack A, or distinguish between hacker-controlled A’s data theft and normal Employee A’s data access, then that intrusion detection fails as well.

Nature of Intrusion Detection

As mentioned previously, intrusion means hackers can operate our assets without consent, with no limits on methods. Identifying the distinctions between intrusion and normal behavior, and separating it from legitimate operations is “intrusion discovery.” In algorithm models, this is classification (intrusion, non-intrusion).

Unfortunately, “black” samples of intrusions are particularly scarce. Training a supervised intrusion detection model with a large set of labeled data to find intrusion patterns is difficult. Thus, intrusion detection strategy developers must invest significant time to refine more precise expression models or expend more effort constructing “intrusion-like” simulated data.

A classic example is, to detect WebShell, security practitioners can search some public WebShell samples on GitHub, totaling around 1000. Yet for machine learning that often requires millions of training data, these are far from enough. Most sample sets on GitHub share similar samples due to a single technical approach, while some adversarial technique samples are missing. Therefore, training aimed at enabling AI to grasp WebShell characteristics and distinguish them is, in principle, unlikely to achieve perfection.

At this time, technical classification of known samples to extract a more precise expression model is called traditional feature engineering. Though typically seen as inefficient repetitive tasks, traditional feature engineering often yields stable results, since a single technical feature can steadily discover a type of WebShell. While creating many malicious samples, with blessings of machine learning and AI, seldom succeeds in practice: Automatically generated samples often depict algorithmic features instead of what WebShell signifies.

On another front, distinguishing between intrusion and legitimate access hinges on whether actions are “unauthorized.” If reinforcement can constrain legitimate access to limited channels and provide strong differentiation for those channels, it can significantly reduce intrusion detection costs. For instance, strictly authenticate access sources – whether human users or program APIs – by requiring valid tokens. Issuing tokens involves multi-dimensional authentication and authorization, with IAM monitoring and recording what each token accesses, allowing collection of lower-level logs for anomaly models.

This full lifecycle risk control model is also foundational to implementing Google’s BeyondCorp zero-trust network.

Thus, two main approaches to intrusion detection emerge:

• Pattern matching using black characteristics (e.g., WebShell keyword matching).
• Compare intrusion behavior against a baseline model of historical business activities (anything black if not white). If historical activity baseline is insufficiently converged, employ reinforcement measures to filter it further, then highlight non-compliant niche behaviors.

 

Intrusion Detection and Attack Vectors

Depending on the target, the attack surface exposed to hackers may differ, and the methods hackers use for intrusion may vary completely. For instance, intruding on our PCs/laptops versus intruding on servers hosted in data centers/cloud differs greatly in attack and defense methods.

For a specific “target,” the channels it accesses form a finite set, with limited attack paths. The combination of “attack methods” + “target attack surface” is called the “attack vector.”

Thus, when discussing intrusion detection model effectiveness, specifying attack vectors and collecting corresponding logs (data) along different attack paths is vital for creating responsive detection models. For example, a dataset from shell commands post-SSH login cannot be used to detect WebShell behavior, and network traffic-derived data cannot sense if hackers executed commands within a post-SSH shell environment.

Hence, if a company claims to have crafted a generalized APT detection model without specifying any scenarios, it’s clearly “boasting.”

Therefore, intrusion detection requires listing various attack vectors, collecting data for each scenario (HIDS+NIDS+WAF+RASP+application layer logs+system logs+PC…), and developing detection models suited to the company’s specific data traits. Different company technical stacks, data scale, and exposed attack surface greatly influence models. For example, many security professionals excel at PHP WebShell detection, but face challenges at a Java-based company…

Common Intrusion Techniques and Countermeasures

Understanding common hacker intrusion methods is crucial to targeted defenses; otherwise, you might easily fall into “politically correct” traps. For instance, a penetration testing team might say, we performed Action A, yet you didn’t notice, which implies a deficiency. The reality is the scenario might be an incomplete intrusion chain, and missing that action might not impact the intrusion detection efficacy. Each attack vector’s potential harm and likelihood of occurrence must be prioritized, along with the cost of resolution and benefits derived—all requiring professional expertise for support and decision-making.

Briefly introduced below are classic steps from hacker tutorial flows (the complete process can be seen in kill chain models):

Before targeting a goal, hackers may not be familiar enough with it, so the first thing they usually do is “reconnaissance,” i.e., information gathering for deeper understanding. For example, hackers need to ascertain the target’s assets (domain names, IPs, services), their statuses, existing known vulnerabilities, their administrators (and how they’re managed legally), and whether any known leaked information exists (e.g., credentials in social engineering databases…).

Once reconnaissance concludes, skilled hackers will devise and gradually test the feasibility of “attack vectors” for various assets; below, common attack methods and defensive advice are listed.

High-Risk Service Intrusion

All public services are “high-risk services” because the protocol or components implementing it may have known attack methods (advanced attackers might even possess matching 0day exploits). If your value is high enough, and hackers have sufficient motivation and resources to exploit, putting high-risk services online, exposed to all, opens a “door” for hackers.

For instance, SSH and RDP services related to management are designed for administrators; anyone with the password/key can log into the server, completing an intrusion. Hackers might obtain credentials via password guessing (complemented by leaks in social engineering databases, cloud storage searches, or brute forcing). This attack type is so prevalent that hackers made it into automated worms scanning the entire internet. A host purchased on a cloud platform with a weak password often gets infected in minutes by worm attacks due to the abundance of automatic attackers.

Perhaps your password setup is very robust, but that’s no reason to keep this service exposed on the internet; we should restrict these ports, allowing access only from our IP (or internal bastion host), cutting off hacker intrusion possibilities.

Similarly, services like MySQL, Redis, FTP, SMTP, MSSQL, Rsync, etc., for server/database/file management shouldn’t be open to the internet without limits. Otherwise, worm-like attack tools may compromise our services within minutes, encrypt data, and even demand Bitcoin for ransomware.

Some high-risk services have RCE vulnerabilities (Remote Command Execution); if left open, hackers can utilize existing exploit tools to GetShell and complete an intrusion.

Defense Suggestion: High costs are involved in conducting intrusion detection for each high-risk service because high-risk services are often diverse with non-universal characteristics. Therefore, reinforcing to limit attack entry points yields higher value. Prohibit all high-risk ports from being opened to the internet, reducing intrusion probability by over 90%.

Web Intrusion

With high-risk port reinforcement, many hacker attack methods from the knowledge base become ineffective. However, Web services are a mainstay for modern internet companies and cannot just be turned off. Consequently, vulnerabilities in dynamic Web services based on PHP, Java, ASP, ASP.NET, Node, CGI in C, etc., become hackers’ primary intrusion entry points.

For instance, utilizing upload features to directly upload a WebShell, leveraging file inclusion features to directly execute remote WebShells (or code), or using code execution features as entry points to run any commands, uploading malicious samples to services parsing images/videos, exploiting parser library vulnerabilities…

Web application security is a specialized field (Dao Ge even wrote a book called “White Hat Talks Web Security”) with matured offensive and defensive scenarios. Of course, since they are all Web-service entry based, intrusion behaviors share certain commonalities. Relatively speaking, we can more easily identify differences between hacker GetShell actions and normal business behavior.

For intrusion trace detection in Web services, consider collecting WAF logs, Access Logs, system call logs recorded by Auditd, or Shell command data and network layer Response data to extract features of successful attacks. Focus most effort on these areas.

0day Intrusion

As leaked toolkits revealed, early NSA enjoyed 0day weapons directly attacking services like Apache and Nginx. This implies adversaries simply didn’t need to care whether our code or services were crafted well—deploying a 0day exploit meant silent GetShell.

Nevertheless, for intrusion detection, this isn’t frightening: regardless of vulnerability entry point adversaries utilize, the Shellcode and subsequent behavior still exhibit commonalities. Whether targeting an Apache zero-day or exploiting a PHP page vulnerability with basic coding issues, from a behavioral perspective, these intrusions may look identical, with models applicable across cases.

Focusing energy on hacker GetShell actions and subsequent behaviors likely holds greater value than focusing on entry point vulnerabilities. Of course, one must practically follow-up with specific vulnerability exploits to confirm expectations.

Office Endpoint Intrusion

Most APT reports depict hackers first targeting individuals (office endpoints), e.g., sending emails to trick us into opening them, taking PC control, engaging in prolonged observation/snooping, securing legitimate credentials, and finally roaming the intranet. Thus, reports primarily focus on Trojan behavior and code family similarities. APT product solutions typically concentrate on system call layers of office endpoints, employing similar methods to examine “undetectable Trojan” behaviors.

Consequently, combining EDR products, email security gateways, office network exit behavior audits, and APT sandbox products allows collection of corresponding data and creation of intrusion detection perception models. Crucially, hackers prioritize internal key infrastructure, including but not limited to AD domain controllers, email servers, password management systems, and privilege management systems—gaining control equates to becoming intranet “deity,” allowing unrestricted actions. Hence, companies must hold targeted attack and defense discussions concerning key infrastructure; Microsoft has even released a specialized hardening white paper regarding AD attacks and defenses.

Basic Principles of Intrusion Detection

A model that cannot thoroughly follow up on every alert is equivalent to an ineffective model. After intrusion occurs, claiming there were alerts that were too numerous or inadequately tracked is “hindsight,” equivalent to lacking discovery capability. Large volumes of daily alerts frustrate security operators.

We must filter out similar recurrent alerts to focus on closing each alert loop. This generates a whitelist or false negatives; thus, false negatives in models are unavoidable.

Since false negatives exist in every model, implementing multiple models across various dimensions and forming deep correlations remains essential. Should WebShell static text analysis be bypassed by hackers, monitoring malicious calls through RASP (Runtime Application Self-Protection) is possible. Accept single model false negatives, yet maintain overall detection ability.

As each scenario model contains false positives and negatives, considerations for “cost-effectiveness” are required when determining necessary and unnecessary scenarios. For instance, highly transformed WebShells made to resemble business code closely may defy human recognition, thus pursuing static text defense in such cases is cost-ineffective; RASP detection solutions offer better cost-effectiveness and feasibility.

We cannot easily know all hacker attack methods or feasibly build strategies for each one (resource constraints considered). Thus, focus on securing key business through hardening techniques (requiring regular monitoring of hardening effectiveness), drastically narrowing attack paths hackers might exploit, and ensuring fallback protection for core business.

Based on the above principles, it becomes apparent that while achieving 100% intrusion detection at a single point may never be possible, utilizing a combination of methods can make it very challenging for attackers to bypass all points.

If bosses or blue teams challenge monitoring capabilities at a single point, investing endlessly to achieve 100% detection often results in futile effort, akin to creating a “perpetual motion machine,” wasting resources with little return. Deploy resources efficiently in placing more in-depth defense chains for superior results.

Mainstream Forms of Intrusion Detection Products

Ultimately, intrusion detection relies on data modeling, such as detecting WebShells involves recognizing web directories, then analyzing files within those directories, necessitating a collector. Whether detecting intrusions based on Shell commands or leveraging system call hooks or Shell interceptions for command acquisition, Network IP reputation/traffic payload detection, or accessing content through email gateways involves integrating into network boundaries and sampling traffic in bypass.

Some comprehensive solutions aggregate multiple sensors, collecting logs across domains, then centrally gathering them in a SOC or SIEM, and allowing big data platforms to conduct integrated analysis. Thus, the industry classifies intrusion detection-related products into the following forms:

• Host Agent Class: Upon a host attack, actions on the host may generate logs, processes, commands, and network traces, prompting host-based intrus
  

  (::post is truncated as the limit has been exceeded::)