In a recent penetration project, the website’s traffic was encrypted using a two-layer encryption algorithm, AES+SM4. When encountering a website with encrypted traffic, it’s usually impossible to perform a vulnerability scan. Manual testing is too inefficient, so I researched a general solution for vulnerability scanning in encrypted transmission scenarios.
How to Encrypt and Decrypt Transmission Traffic?
There are usually two ways to achieve this.
Reimplement Encryption and Decryption Algorithms with Python
If the website’s encryption algorithm is a standard one like AES, it can be directly implemented using Python’s pycrypto module. For relatively simple encryption algorithms, you can extract the encryption and decryption code from JavaScript and execute it with Python’s execJs to get the result.
For example, manually implementing the website’s encryption and decryption code with Python is as follows:
This method can only handle simple encryption algorithms and requires writing code, which is relatively cumbersome.
JSRPC Forwarding Encryption and Decryption Functions
If you encounter complex encryption algorithms, such as those modified from standard algorithms, where it’s impossible to extract key code from JavaScript, using jsrpc to directly call the encryption and decryption functions in the browser is very suitable.
https://github.com/jxhczhl/JsRpc
Since the website’s transmission content is encrypted, the traffic needs to be decrypted before entering the passive scanner.
Mitmproxy for Traffic Encryption and Decryption
mitmproxy is a proxy used for MITM (Man-in-the-middle attack). A proxy used for MITM first forwards requests like a normal proxy, and can also inspect, record intercepted data, or tamper with data to trigger specific behaviors on the server or client. Unlike packet capture tools like fiddler or wireshark, mitmproxy can not only intercept requests to help developers view and analyze but also perform secondary development through custom scripts.
As mentioned above, you can reimplement the encryption and decryption algorithms with Python, but before the traffic enters the passive scanner, a middleman is needed to perform encryption and decryption operations, and mitmproxy plays this role. You can run the Python encryption and decryption code in mitmproxy, install a trusted mitmproxy certificate in the system, and let mitmproxy decrypt the HTTPS traffic before performing secondary decryption.
mitmproxy Command Line
The first mitmproxy command line needs to set the upstream proxy to the IP and port listened by xray.
mitmdump -s code.py -p 8010 --set block_global=false --mode upstream:http://xray-ip:port --ssl-insecure
The second mitmproxy command line forwards directly to the target website.
mitmdump -s code.py -p 8020 --set block_global=false --ssl-insecure
mitmproxy Code Template
Since the process involves four times of mitmproxy encryption and decryption, two mitmproxy listening ports need to be opened to forward traffic.
First Traffic Encryption and Decryption Code Template
from mitmproxy import http
# Encryption and decryption functions can be reimplemented with Python or forwarded directly using jsrpc
# Encryption function
def myEncrypt(data):
pass
# Decryption function
def myDecrypt(data):
pass
# Decrypt the request content from the browser and forward it to the scanner
def request(flow: http.HTTPFlow) -> None:
if flow.request.pretty_host != "target.com":
return # If it's not the target domain, forward directly without any operation
try:
param = flow.request.content.decode()
decryptData = myDecrypt(param)
print("First request decryption:\n"+decryptData)
# Replace the encrypted message with the decrypted message
flow.request.content = decryptData
except Exception as e:
print(f"Message decryption failed: {e}")
# Encrypt the response content from the scanner and forward it to the browser
def response(flow: http.HTTPFlow) -> None:
if flow.request.pretty_host != "target.com":
return # If it's not the target domain, forward directly without any operation
try:
param = flow.response.content.decode()
encryptData = myEncrypt(param)
print("First response encryption:\n" + encryptData)
# Encrypt the plaintext message for transmission
flow.response.content = encryptData
except Exception as e:
print(f"Message encryption failed: {e}")
Second Traffic Encryption and Decryption Code Template
from mitmproxy import http
# Encryption and decryption functions can be reimplemented with Python or forwarded directly using jsrpc
# Encryption function
def myEncrypt(data):
pass
# Decryption function
def myDecrypt(data):
pass
# Encrypt the request content from the scanner and forward it to the website
def request(flow: http.HTTPFlow) -> None:
if flow.request.pretty_host != "target.com":
return # If it's not the target domain, forward directly without any operation
try:
param = flow.request.content.decode()
encryptData = myEncrypt(param)
print("Second request encryption:\n" + encryptData)
# Replace the plaintext message with the encrypted message
flow.request.content = encryptData
except Exception as e:
print(f"Message encryption failed: {e}")
# Decrypt the response content from the website and forward it to the scanner
def response(flow: http.HTTPFlow) -> None:
if flow.response.pretty_host != "target.com":
return # If it's not the target domain, forward directly without any operation
try:
param = flow.response.content.decode()
decryptData = myDecrypt(param)
print("Second request decryption:\n"+decryptData)
# Replace the encrypted message with the plaintext message
flow.response.content = decryptData
except Exception as e:
print(f"Message decryption failed: {e}")
xray Configuration Modification
Since mitmproxy performs encryption and decryption on traffic before and after xray, xray only needs to configure the upstream proxy. However, two configurations need to be modified, both http and upstream_proxy should be changed to the IP and port of mitmproxy.
Practical Testing
Although the penetration testing process in encrypted transmission scenarios is relatively complex, the traffic itself is encrypted, so the WAF cannot intercept it, saving some trouble.
In practice, using the mitmproxy template I provided and replacing the encryption and decryption functions, you can directly perform vulnerability scanning on websites with encrypted transmission. Job done!