Improving Network Monitor (and Message Analyzer) dissection support in Wireshark

[Michael Mann via Wireshark-dev <wireshark-dev () wireshark org>]

I started working on the problem of having Wireshark read packet comments from Network Monitor (NetMon) (.cap) files. 
It escalated quickly and I found myself implementing way more functionality than I had intended just because of the 
information I stumbled across (and being "in the neighborhood").  I thought it was worth sharing some of that 
information on the -dev mailing list to give guidance in case anyone else wants to expand Wireshark's support of Netmon 
files. I do not have any plans to further expand the dissection, but will gladly answer any questions if others want to 
give it a try. Note this is only related to .cap files.  Support for .etl files has NOT been added (see bug 6694), and 
I'm not sure how much of this information may apply to .etl files.

.cap files use NPL (NetMon Parsing Language?), which is a combination of a C-like language mixed with IDL.  It is 
certainly "human readable", but I think the meaning of some of the "IDL" parts would make it a bit too complex to make 
it worthwhile to have a "npl2wrs" tool.  However, I did notice what appears to be some work towards that in the 
Wireshark tools directory.  Jakub, any comments?
Message Analyzer appears to use NPL files, but also OPN files, which seem to have a similar format.  Not sure of the 
details/differences, but I used both for reference when adding to Wireshark functionality.

Some of the functionality I added to Wireshark for .cap support was in wiretap.  Most of the rest, while done with 
dissectors, could be considered "metadata" and may be "translated" into pcapng blocks.  But since .cap files were 
treating them as "frames", I did as well as that's why they ended up with dissectors.  One of the key "frame types" 
would be "NetMon events".  It has an architecture that aligns with using a Wireshark dissector table.  I believe the 
"event frame types" can hold protocol dissection data, but I focused on the "metadata" since that's what was present in 
the captures I had (retrieved from Bugzilla)
I stayed away from protocol dissection mostly because I didn't have capture files for it, but also because I know 
Wireshark is already pretty good at that.  I was focused on how to "connect the dots" so that data could easily be 
passed to existing (protocol) dissection.

My two key sources of information were:
1. https://NMParsers.svn.codeplex.com/svn (just grab it with SVN client).  It's basically the "parser source" for 
NetMon. There are 2 directories, "Develop_Branch" and "Verified_Build_Branch".  For what I've done, I haven't seen a 
difference between the two, but I've also not done a full compare.  Given that I believe work on Network Monitor has 
stopped, these may not be updated anymore.
2. C:\Program Files\Microsoft Message Analyzer\OPNAndConfiguration.  This is the parser information installed with 
MessageAnalyzer.  Bug 10556 (https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=10556) seems to give some information 
on how to extract the information without installing Message Analyzer, but since I'm mostly a Windows developer, this 
wasn't an issue for me.

To me both are comparable to Wireshark's dissector directory (with a little bit of wiretap mixed in to handle 
"metadata").

Sharkbytes:
1. frame.npl, netmon.npl and MAExported.npl (from NMParsers\Develop_Branch\NPL\common directory is information for 
wiretap (netmon.c).  I implemented what I thought was "most popular" and things that I found sample captures for.

2. grep "NetEvent.UserData" in the NMParsers directory and you'll see all of the potential GUIDs used for Provider IDs. 
 Provider IDs are the "identifier" to the layer following a "NetMon event".  Again, I focused on "metadata" dissectors, 
but there were over 100 GUIDs registered with "NetEvent.UserData", most of which are probably protocol related.

3. grep "EventDescriptor.DefaultKeyword" for a potential match with a GUID from NetEvent.UserData.  The "NetMon event 
dissector" displays the keyword (and a few other fields), but its the subsequent layer that really defines its meaning.

4. packet-netmon.c was used mostly for functionality I found from NPL code, packet-messageanalyzer.c was for code I 
associated with MessageAnalyzer (either from OPN files or NPL files that used "MA" or other indications to show it was 
for MessageAnalyzer).  Protocol dissection may go in existing dissectors (if related/similar) or there may be new 
protocols that justify new dissection files. packet-netmon.c, packet-messageanalyzer.c, and packet-usb.c can be used as 
current examples of how to hook into the ProviderID dissector table ("netmon.provider_id") used by "NetMon events"

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe