Integrating Zeek ELK Stack: A Step-by-Step Guide to Network Traffic Monitoring with GrayLog

1. Introduction to Integrating Zeek ELK Stack: An Open Source IDS for Network Traffic Analysis and Monitoring

 The Zeek Network Security Monitor: A powerful framework for network traffic analysis and security monitoring.

Integrating Zeek ELK Stack: Zeek Network Security Monitoring is a powerful framework for analyzing network traffic and monitoring security.

• 1. In-depth Analysis: Zeek comes with analyzers for a wide range of protocols, allowing high-level semantic analysis at the network application’s layer.
• 2. Highly Adaptable and Flexible: Zeek’s domain-specific scripting language enables site-specific monitoring policies and is not limited to any specific detection method.
• 3. Efficiency: Zeek targets high-performance networks and is used and run on many large-scale sites.
• 4. Advanced Statefulness: Zeek maintains extensive states concerning the network application layer it monitors and provides advanced logging of network activities.

Official Documentation for Integrating Zeek ELK Stack

 https://github.com/zeek/zeek
https://docs.zeek.org/

Reference Links for Integrating Zeek ELK Stack

 https://linux.cn/article-14770-1.html
http://www.senlt.cn/article/171944181.html
https://blog.csdn.net/u013613428/article/details/110728269

After reading the article “Integrating Zeek with the ELK Stack,”

decided to attempt sending Zeek’s network traffic monitoring logs to GrayLog and displaying them there.

Overview of the implementation plan and problems to be solved

• 1. Deployment of Zeek (Can it be deployed using RPM packages? Compilation deployment is too time-consuming.)
• 2. Traffic Mirroring Problem: How to configure the network card of the Zeek server in promiscuous mode, and how to configure the switch’s traffic mirroring.
• 3. Log Reading Problem of Zeek Traffic Analysis: Convert Zeek logs to JSON format logs, use graylog-sidecar+filebeat to read them, and extract field information using a JSON extractor.
• 4. Queries on GrayLog, Dashboard Configuration

The specific implementation process is as follows

2. Download and Deploy Zeek’s CentOS7 RPM Package using the Integrating Zeek ELK Stack

 https://zeek.org/get-zeek/

Due to network speed issues, I manually downloaded the necessary RPM dependencies for Zeek beforehand.

1. Installing Zeek with Integrating Zeek ELK Stack

 yum localinstall *.rpm

2. Integrating Zeek ELK Stack: Add /opt/zeek/bin Directory to the PATH Environment Variable

 echo "PATH=$PATH:/opt/zeek/bin" > >  /etc/profile
source /etc/profile
echo $PATH

3. Set the Network Card to Promiscuous Mode for Integrating Zeek ELK Stack

 ifconfig ens33 promisc 

The operation to configure mirroring traffic on the switch is not demonstrated here. Please explore on your own by referring to the article at this link and resolve it according to your actual network environment.

 https://blog.csdn.net/u013613428/article/details/110728269

4. Edit node.cfg for Integrating Zeek ELK Stack

 cd /opt/zeek/etc
vim node.cfg
Modify the interface to the actual network card in use

5. Edit local.zeek

 cd /opt/zeek/share/zeek/site/
vim local.zeek 

Add the following lines:
@load policy/tuning/json-logs.zeek

6. Start Zeek

 zeekctl 
[ZeekControl] >  deploy

7. View Zeek’s Traffic Analysis Logs

 cd /opt/zeek/logs/
cd current/
ll

Logs are currently in JSON format.

3. Deploy Graylog-sidecar + Filebeat to Read Zeek’s Logs

1. Download and Install Graylog-sidecar and Filebeat RPM Packages

 https://github.com/Graylog2/collector-sidecar/releases
https://mirrors.cloud.tencent.com/elasticstack/7.x/yum/7.10.2/filebeat-7.10.2-x86_64.rpm

rpm -ivh filebeat-7.10.2-x86_64.rpm 
rpm -ivh graylog-sidecar-1.3.0-1.x86_64.rpm 

2. Create Sidecar Token and Modify Sidecar Configuration

 vim /etc/graylog/sidecar/sidecar.yml
Modify the following five configurations:
server_url: "http://192.168.31.170:9000/api/"
server_api_token: "hkoa0tv9gl9u09drhtckjbdv90v98v5mor5jropo0p0741oib3g"
node_name: "zeek"
update_interval: 10
send_status: true

3. Start Graylog-sidecar and Deploy Configuration

 graylog-sidecar -service install
graylog-sidecar -service start

4. Configuring Beat Type Input on Graylog

Open port 5044 in the firewall.

firewall-cmd --permanent --zone=public --add-port=5044/tcp
firewall-cmd --reload

5. Confirm if logs are received

4. Configuring JSON Format Field Extraction and Setting Up a Dashboard on Graylog

5. Conclusion

1. The RPM packages used in this article have been fully packaged. Download links are as follows:

Link: https://share.weiyun.com/knIqb1EA 
Password: bk2un9

2. Extension

If possible, Zeek traffic logs can be correlated with the MicroStep API interface for querying. This enables abnormal DNS and other traffic detection and alerts.

Refer to previous articles.

Using HTTP JSONPath in GrayLog to Invoke MicroStep Online Cloud API to Identify Threat IPs