Optimizing Zeek Network Analyzer Performance with PF_RING and RSS Configuration

Introduction

Zeek

Zeek is an open-source network traffic analyzer widely used as a Network Security Monitor (NSM) to aid in investigating suspicious or malicious activities. In addition to its security functions, Zeek network analyzer is also capable of handling various traffic analysis tasks such as performance profiling and troubleshooting.

PF_RING

PF_RING is a new type of network socket that can significantly improve packet capture speed compared to libpcap. If you use the PF_RING ZC (zero copy) driver, it can achieve extremely high packet capture/transmission speeds, with PF_RING ZC capable of 100G packet capture under sufficient hardware conditions.

Installation Guide

Installing PF_RING

Reference: PF_RING installation and usage guide

Installing Zeek

Dependency Installation

For CentOS7

  sudo yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python3 python3-devel swig zlib-devel

(Note: CentOS requires a version of cmake 3.0 or above)

Ubuntu/Debian

 sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python3 python3-dev swig zlib1g-dev

Source Code Download

 git clone --recursive https://github.com/zeek/zeek

Compilation

 cd zeek-X.X.X
./configure --with-pcap=/usr/local/lib
make && sudo make install

Ensure Zeek is correctly linked to the pf_ring-aware libpcap:

 ldd /usr/local/zeek/bin/zeek | grep pcap

Configuration File

The configuration file is located at /usr/local/zeek/etc/node.cfg

Write the following configuration file:

 [logger-1]
type=logger
host=localhost

[manager]

type=manager host=localhost

[proxy-1]

type=proxy host=localhost

# Configure worker, multiple workers can be configured simultaneously

 [worker-1]
type=worker
host=localhost    
interface=ens33  
lb_method=pf_ring
lb_procs=4
pin_cpus=0,1,2,3
#[worker-2]
#type=worker
#host=192.168.0.100
#interface=eth0

Note:

worker- is the node for actual traffic collection

host- Traffic collection node IP address

interface- Interface name

lb_method- Packet capture method

lb_procs- Number of load balancing queues

pin_cpus- The number of CPUs bound is usually the same as the number of load balancing queues

The above is a cluster operation mode, where multiple computing nodes can work together. Here it’s configured to run locally, so both the worker and master are set to localhost (Note: When using a cluster, you only need to write a configuration file on the manager. Of course, the worker nodes also need to have Zeek installed). Worker can be configured for other nodes, but it requires password-less SSH login to other worker nodes from the manager node. Method for configuring SSH password-less login.

lb_procs Parameter Detailed Explanation

The lb_procs parameter indicates the number of load-balanced queues for the monitoring interface, typically seen in two situations.

• Using RSS NIC Multi-queue Technology

RSS (Receive Side Scaling) is a network card driver technology designed to efficiently distribute incoming packets across multiple CPUs in a multi-processor system.

Almost all Intel (and other vendor) NICs support RSS, which means they can hash packets in hardware to distribute the load across multiple RX queues. In some cases, when RSS is unavailable or insufficiently flexible (e.g., needing custom distribution functions), ZC can be used to replace it with software distribution.

Suppose we configured the interface ens33 with 4 RSS queues; you would configure lb_procs=4 and correspondingly configure CPUs to handle traffic from the four queues: pin_cpus=0,1,2,3.

RSS is a load-balancing mechanism designed to handle high volumes of traffic, greatly enhancing traffic processing capability when employed.

Refer to the RSS configuration method.

Using PF_RING ZC for Traffic Distribution

PF_RING ZC can distribute traffic at high speeds from the network card using software to distribute it to different queues. Similar to RSS functionality, but implemented via software methods.

 sudo zbalance_ipc -i zc:eth1 -c 99 -n 8 -m 1 -g 8

-c 99 is the cluster ID

-n 8 is the number of queues

-g 8 is the CPU binding for zbalance_ipc

You should use zc:cluster id as the interface name, as shown in the example below.

 [logger-1]
type=logger
host=localhost

[manager]

type=manager host=localhost

[proxy-1]

type=proxy host=localhost # Configure worker, multiple workers can be configured simultaneously

[worker-1]

type=worker host=localhost interface=zc:99 lb_method=pf_ring lb_procs=8 pin_cpus=0,1,2,3,4,5,6,7 #[worker-2] #type=worker #host=192.168.0.100 #interface=eth0

Starting Zeek

Enter the following command in the terminal to access the Zeek console:

 /usr/local/zeek/bin/zeekctl

First use the install command to load the configuration, then use start, stop commands to start and stop Zeek.

 [ZeekControl] >  install 
[ZeekControl] >  start
[ZeekControl] >  stop

Accelerating with PF_RING ZC

PF_RING™ ZC (Zero Copy) is a flexible packet processing framework that allows you to achieve line-rate packet processing (RX and TX) at 1/10 Gbit under any packet size, with potentially higher speeds on sufficient hardware. It considerably boosts efficiency by bypassing the kernel protocol stack and directly capturing packets from the network card. Using ZC requires installation of specific drivers, with installation methods available in the PF_RING installation and usage guide.

In Zeek, using ZC is straightforward; simply use the format zc:, with an example configuration below:

 [worker-2]
type=worker
host=192.168.0.101
lb_method=pf_ring
interface=zc:eth0
lb_procs=8
pin_cpus=0,1,2,3,4,5,6,7

Accelerating with PF_RING FT

Using PF_RING FT can filter out traffic you do not wish to monitor, thus achieving acceleration. It’s important to note the difference between PF_RING FT and PF_RING ZC acceleration. PF_RING ZC optimizes acceleration during packet capture, while PF_RING FT uses filtering to disregard unnecessary traffic post-capture, focusing only on relevant traffic for examination, thereby achieving acceleration. PF_RING FT is usually used alongside PF_RING ZC but can also be used with libpcap. The method is as follows.

Write the PF_RING FT configuration file /etc/pf_ring/ft-rules.conf

File contents:

 [filter]
YouTube = discard
Netflix = discard

Above is a simple filter rule example that discards YouTube and Netflix traffic, as PF_RING FT uses DPI technology to directly identify and filter application layer traffic.

Zeek Configuration File Changes

Simply add the following line under the corresponding worker configuration in the Zeek configuration file:

 env_vars=PF_RING_FT_CONF=/etc/pf_ring/ft-rules.conf

For example:

 [worker-2]
type=worker
host=192.168.0.101
lb_method=pf_ring
interface=zc:eth0
lb_procs=8
pin_cpus=0,1,2,3,4,5,6,7
env_vars=PF_RING_FT_CONF=/etc/pf_ring/ft-rules.conf

Original Statement: This article is authorized by the author to be published in the Tencent Cloud Developer Community. Reproduction without permission is prohibited.

If there is any infringement, please contact [email protected] for removal.

Enterprise Organization Enterprise Network Security