Mastering IoT Traffic Analysis: Enhancing Wireshark with MQTT over WebSocket Parsing

1. Background

The expansion of the Internet of Things (IoT) has significantly increased the volume and variety of data

Recently, during IoT traffic analysis, it was discovered that when apps use the MQTT protocol, they often communicate with the server using SSL+WebSocket+MQTT. After intercepting data via an SSL MITM, Wireshark cannot automatically parse the MQTT semantics and can only parse up to the WebSocket layer, as shown in the picture. Although WebSocket data with masks removed is displayed, analyzing MQTT is still quite challenging. Therefore, I intend to write a plugin leveraging Wireshark’s built-in MQTT parsing feature to analyze the data in the Data section instead of writing a completely new parser from scratch. Note: Many tutorials teach how to add a new protocol, such as setting protocol attributes. It’s recommended to refer to [2]. This article mainly organizes the logic of writing a plugin.

/>

2. Basics of Writing a Wireshark Plugin with Lua

 Previous experts have introduced basic tutorials on writing Wireshark plugins with Lua, which you can refer to at the end of this article [1][2]. Here, I summarize my understanding, as there truly isn’t a document that gave me a thorough understanding from beginner to expert.

1. First, you need to know the relevant concepts of dissectors and post-dissectors [3]

1) A dissector is used to be called by Wireshark to parse packets or parts of packets. It must be registered as a Proto object to be invoked by Wireshark. At the same time, we can use Wireshark’s built-in dissectors. An example of registering a dissector is shown below.

Code Language: javascriptCopy

-- trivial protocol example-- declare our protocol--trival is the protocol name, followed by the description, both need to be unique in Wireshark.trivial_proto = Proto("trivial","Trivial Protocol")-- create a function to dissect itfunction trivial_proto.dissector(buffer,pinfo,tree)    pinfo.cols.protocol = "TRIVIAL"    local subtree = tree:add(trivial_proto,buffer(),"Trivial Protocol Data")    subtree:add(buffer(0,2),"The first two bytes: " .. buffer(0,2):uint())    subtree = subtree:add(buffer(2,2),"The next two bytes")    subtree:add(buffer(2,1),"The 3rd byte: " .. buffer(2,1):uint())    subtree:add(buffer(3,1),"The 4th byte: " .. buffer(3,1):uint())end-- load the udp.port tableudp_table = DissectorTable.get("udp.port")-- register our protocol to handle udp port 7777udp_table:add(7777,trivial_proto)

2) There are many ways to register a dissector; you can use the function register_postdissector(trivial_proto) to register as a post-dissector, which executes after all dissectors have run; or you can register on the DissectorTable, which allows using results from previous layer protocols already parsed by Wireshark. For example, if your dissector wants to parse a protocol on TCP port 7777, you can use the following code without starting from the TCP or IP layer.

Code Language: javascriptCopy

-- load the udp.port tableudp_table = DissectorTable.get("udp.port")-- register our protocol to handle udp port 7777udp_table:add(7777,trivial_proto)

This feature is very powerful. Intuitively, if you want to parse the MQTT protocol on WebSocket, you can write like this [6] (but for some reason, I’ve never been able to parse it successfully this way):

Code Language: javascriptCopy

local mqtt_dissector = Dissector.get("mqtt")local ws_dissector_table = DissectorTable.get("ws.port")ws_dissector_table:add(8083, mqtt_dissector)

From the code above, we learn to directly use the method Dissector.get in Wireshark to get the dissector. More methods can be found in the official documentation Chapter 11 [7], such as how to get all supported protocols? Is the keyword for the MQTT protocol dissector in uppercase or lowercase? You can write it like this [8]:

Code Language: javascriptCopy

local t = Dissector.list()for _,name in ipairs(t) do    print(name)end--view all supported tableslocal dt = DissectorTable.list()for _,name in ipairs(dt) do    print(name)end

3) When called, Wireshark will pass the dissector three parameters: the data buffer (a Tvb object [4]), packet information (Pinfo object [5]), and the tree structure displayed graphically (TreeItem object). Understanding these three parameters is essential, and note that they are not native data types of Lua, often requiring method calls within objects to convert them. Through these three parameters, the dissector can obtain and modify packet-related information.

Tvb is the packet’s data content, and you can extract content like this. Often, we need to extract the packet’s content for string processing or provide a string converted into Tvb for the dissector to handle, requiring some conversion as shown in the following code [10], further details are available in [9].

Code Language: javascriptCopy

local b = ByteArray.new(decipheredFrame)local bufFrame = ByteArray.tvb(b, "My Tvb")

Pinfo is often interpreted as message information. Personally, I simply understand it as having an interface for accessing packets in the manner shown in the figure, with the most common example being modifying the protocol column name or the message displayed in the info column, such as pinfo.cols.protocol = “MQTT over Websocket”, more properties can be obtained from reference [5].

TreeItem object represents a tree node in the packet resolution tree, and with it, nodes can be dynamically added to the graphical interface.

2. Debug and Enable Plugins

Start

Wireshark loads the init.lua script when starting, found in the Wireshark installation directory on Windows and in etc/wireshark on Linux. To execute your written plugin, simply add dofile(“.\\plugins\\mqttoverwebsocket.lua”) at the end of this script to execute it. The shortcut for reloading Lua scripts is Ctrl+Shift+L.

Debug

If the script has syntax errors, Wireshark’s GUI will prompt during loading; if there are runtime errors, they’ll display within the graphical protocol tree; Wireshark also has a Lua terminal for executing written plugin scripts, and printing error messages can be opened through “Tools—Lua—console”, to dynamically execute scripts through “Tools—Lua—evaluate”. Note that to see the output, you need to use Wireshark’s built-in functions such as debug(text) to output [14].

3. Implement Parsing of MQTT Protocol on WebSocket

 For unknown reasons, registering the MQTT protocol dissector to ws.port or ws.protocol still cannot automatically parse MQTT, so I chose to first obtain the already parsed WebSocket data field with the mask removed and then convert it to TVB for automatic parsing by the MQTT dissector. The method of obtaining content post-packet parsing mainly references the example in [11] and [12], using the fieldinfo class and global function all_field_infos() to get content from different parts of the parsing tree.

Since the tree passed into the MQTT dissector is the root of this packet, a node will also be automatically added. This finally achieved a satisfactory effect.

Code Language: javascriptCopy

do    -- calling tostring() on random FieldInfo's can cause an error, so this func handles it    local function getstring(finfo)        local ok, val = pcall(tostring, finfo)        if not ok then val = "(unknown)" end        return val    end        -- Create a new dissector    MQTToverWebsocket = Proto("MQTToverWebsocket", "MQTT over Websocket")    mqtt_dissector = Dissector.get("mqtt")    -- The dissector function    function MQTToverWebsocket.dissector(buffer, pinfo, tree)        local fields = { all_field_infos() }        local websocket_flag = false        for i, finfo in ipairs(fields) do            if (finfo.name == "websocket") then                websocket_flag = true            end            if (websocket_flag == true and finfo.name == "data") then                local str1 = getstring(finfo)                local str2 = string.gsub(str1, ":", "")                local bufFrame = ByteArray.tvb(ByteArray.new(str2))                mqtt_dissector = Dissector.get("mqtt")                --mqtt_dissector:call(finfo.source, pinfo, tree) #9 BUG                mqtt_dissector:call(bufFrame, pinfo, tree)                --mqtt_dissector:call(finfo.value, pinfo, tree)                websocket_flag = false                pinfo.cols.protocol = "MQTT over Websocket"            end    end                --ws_dissector_table = DissectorTable.get("ws.port")        --ws_dissector_table:add("443",mqtt_dissector)    end    -- Register the dissector    --ws_dissector_table = DissectorTable.get("ws.port")    --ws_dissector_table:remove(443, mqtt_dissector)    --ws_dissector_table:add(443, MQTTPROTO)    --ws_dissector_table:add_for_decode_as(mqtt_dissector)    register_postdissector(MQTToverWebsocket)end

By ascii0x03, 2018/4/10, please credit the source if reproduced.