Monday, September 7, 2009

The Network Monitor API: Part II

The LoadCapAndFilter example from Network Monitor Examples from Codeplex allows you to specify a particular filter from Network Monitor API. Some fragments from the code are below. Note how the string is escaped (e.g. \"GET\") :

[LoadCapAndFilter.cpp]
/Add filter
ret = NmAddFilter(myFrameParserConfig, L"http.request.command == \"GET\"", &myHTTPFilterID);
...
//Add field
ret = NmAddField(myFrameParserConfig, L"http.request.uri", &myHTTPFieldID);
....
// Obtain the value of http.request.uri from frame. We
// know that strings are passed as word pointer to unicode string in the variant.
..
ret = NmGetFieldValueString(myParsedFrame, myHTTPFieldID, 256, (LPWSTR)value);

Sample output:

LoadCapAndFilerHTTP.exe Miscellaneous_001.cap
sparser.npb:001.000 Successfully unserialized NPL parser 'C:\Users\Admin\AppData\Local\Microsoft\Network Monitor 3\sparser.npb.
Frame 14: HTTP: /crls/globalca1.crl
Frame 100: HTTP: /crls/globalca1.crl
Frame 227: HTTP: /crls/globalca1.crl
Frame 547: HTTP: /crls/globalca1.crl
....

I can modify the LoadCapAndFilter Network Monitor example to parse as needed. For example, Microsoft has a global load balacer that it contacts both before and after secondary authorization for Wireless Network Connections. It functions by contacting http://nssi.glbdns.microsoft.com/ncsi.txt and checking to see is a successful http request is returned. If it can't do so, the returned payload shows no http status code:

HTTP HTTP:Response, HTTP/1.0, Status Code = , URL: /ncsi.txt

If it gets a hit, this request shows:

HTTP:Response, HTTP/1.1, Status Code = 200, URL: /ncsi.txt

By changing the top code to (far) below, I can cycle through all my wireless sniffs to see how many times my Vista laptop tries to get: http://nssi.glbdns.microsoft.com/ncsi.txt  with :

for /f %i in ('dir /b *.cap') do LoadCapAndFilterGet_NCSI.exe %i 

[output]:
LoadCapAndFilterGet_NCSI.exe Test.cap
sparser.npb:001.000 Successfully unserialized NPL parser 'C:\Users\Admin\AppData\Local\Microsoft\Network Monitor 3\sparser.npb.
Frame 8012: HTTP: /ncsi.txt
Frame 8452: HTTP: /ncsi.txt
Frame 8815: HTTP: /ncsi.txt
Frame 9178: HTTP: /ncsi.txt
Frame 9560: HTTP: /ncsi.txt
...

[LoadCapAndFilterGETNCSI.cpp]:

//Add filter
ret = NmAddFilter(myFrameParserConfig, L"HTTP.Request.URI == \"/ncsi.txt\"", &myHTTPFilterID);
...
//Add field
ret = NmAddField(myFrameParserConfig, L"http.request.uri", &myHTTPFieldID);
....


Another example:

This code ...


//Add filter
ret = NmAddFilter(myFrameParserConfig, L"TCP.Port == 443", &myHTTPFilterID);
if(ret != ERROR_SUCCESS)
{
wprintf(L"Fail to load Add fitler, error: \n", ret);
}

//Add field
ret = NmAddField(myFrameParserConfig, L"SSL.TlsRecLayer.TlsRecordLayer.SSLHandshake.HandShake.ClientHello.Extns.ClientHelloExtension.ServerNameList.ServerName", &myHTTPFieldID);
if(ret != ERROR_SUCCESS)
{
wprintf(L"Fail to load Add field, error: \n", ret);
}

produces this output:

C:\Users\Admin\Documents\Network Monitor 3\Captures>LoadCapAndFilterTCP001.exe Test.cap
sparser.npb:001.000 Successfully unserialized NPL parser 'C:\Users\Admin\AppData\Local\Microsoft\Network Monitor 3\sparser.npb.
Frame 7160: TCP443: www.google.com
Frame 16097: TCP443: signin.evri.com
Frame 16341: TCP443: signin.evri.com
Frame 16577: TCP443: www.connect.facebook.com
Frame 16591: TCP443: www.connect.facebook.com
Frame 16599: TCP443: www.connect.facebook.com
Frame 16631: TCP443: www.connect.facebook.com

..

To make LoadCapAndFilter work, the correct return types for  NMGetFieldValue must be assigned. I had quite a few problems making other queries work, seemingly because of this.


// Obtain the value of http.request.uri from frame. We
// know that strings are passed as word pointer to unicode string in the variant.
WCHAR value[256];
ret = NmGetFieldValueString(myParsedFrame, myHTTPFieldID, 256, (LPWSTR)value);
  if(ret == ERROR_SUCCESS

Tuesday, September 1, 2009

The NetworkMonitor API: Part I

I've spent the last three weeks building the Network Monitor Examples from Codeplex: http://nmexperts.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=27988.  Sniffers have been pretty black box to me before this project. I was prompted to do this because Network Monitor 3.3 on 64 bit systems doesn't produce captures that can be analyzed by logparser.exe.  This is good and bad.  Logparser only dumped out 20 fields from netmon *.cap files. Despite the struggle, it was worth installing the latest versions (VS2008, VS2009 Express ), configuring VS C++ to work with the WDDK and the Netmon API and compiling the examples on both 32 and 64 bit systems. Microsoft has released the Netmon SDK and API to the web at codeplex.com.  Network Monitor itself is a free download and the lib and header files come along for the ride. Open Parsers are part of the project, allowing the coder to create his own parsers; filters; experts.

The samples allow you to build open, close, save, filter and parse captures files and parsers. Some examples are below. This project is best done by someone unafraid of Visual Studio and the WDDK.

IterateFields.exe Test.cap 500
sparser.npb:001.000 Successfully unserialized NPL parser 'C:\Users\Admin\AppData\Local\Microsoft\Network Monitor 3\sparser.npb.
Iterate the fields of frame #500
Frame.WiFi (WiFi) - Offset: 0, Size: 1536
 WiFi.WiFi.MetaData (WiFiMetadata) - Offset: 0, Size: 32
  WiFi.WiFi.MetaData.Version (UINT8) - Offset: 0, Size: 1
  WiFi.WiFi.MetaData.Length (UINT16) - Offset: 1, Size: 2
  WiFi.WiFi.MetaData.OpMode (UINT32) - Offset: 3, Size: 4
  WiFi.WiFi.MetaData.OpMode.StationMode (UINT32) - Offset: 3, Size: 0
  WiFi.WiFi.MetaData.OpMode.APMode (UINT32) - Offset: 3, Size: 0
  WiFi.WiFi.MetaData.OpMode.ExtensibleStationMode (UINT32) - Offset: 3, Size: 0
  WiFi.WiFi.MetaData.OpMode.Unused (UINT32) - Offset: 3, Size: 3
  WiFi.WiFi.MetaData.OpMode.MonitorMode (UINT32) - Offset: 6, Size: 0
  WiFi.WiFi.MetaData.Flags (UINT32) - Offset: 7, Size: 4
  WiFi.WiFi.MetaData.PhyType (UINT32) - Offset: 11, Size: 4
  WiFi.WiFi.MetaData.Channel (UINT32) - Offset: 15, Size: 4
  WiFi.WiFi.MetaData.lRSSI (INT32) - Offset: 19, Size: 4
  WiFi.WiFi.MetaData.Rate (UINT8) - Offset: 23, Size: 1
  WiFi.WiFi.MetaData.TimeStamp (FILETIME) - Offset: 24, Size: 8 ....

IterateFieldsWithDisplayFormat.exe  Test.cap 500
sparser.npb:001.000 Successfully unserialized NPL parser 'C:\Users\Admin\AppData\Local\Microsoft\Network Monitor 3\sparser.npb.
Iterate the fields of frame #500
Field count = 92
WiFi: [Unencrypted Data] .T...., (I)

Error 1168 tryin to retreive display name for frame 499 element 1. Version: 2 (0x2)
  Length: 32 (0x20)
  OpMode: Extensible Station Mode
  StationMode: (...............................0) Not Station Mode
  APMode: (..............................0.) Not AP Mode
  ExtensibleStationMode: (.............................1..) Extensible Station Mode
  Unused: (.0000000000000000000000000000...)
  MonitorMode: (0...............................) Monitor Mode
  Flags: 4294967295 (0xFFFFFFFF)
  RemData: Outbound
  TimeStamp: 08/18/2009, 05:41:19 PM

FrameControl: .T.... (0x0801)
Version: (..............00) 0
Type: (............10..) Data
SubType: (........0000....) Data
DS: (......01........) STA to DS via AP
MoreFrag: (.....0..........) No
Retransmission: (....0...........) No
PowerMgt: (...0............) Active Mode
MoreData: (..0.............) No
Encrypted: (.0..............) No
Order: (0...............) Unordered....

GetFrameComments 100secwithComments.cap
Frame 1 Comment Info:
  TitleByteLength: 34, Title: Test Comment 001
  DescriptionByteLength: 137, Description: {\rtf1\ansi\ansicpg1252\deff0\deflang1033{\fonttbl{\f0\fnil\fcharset0 MS Shell D
\viewkind4\uc1\pard\f0\fs17 testing...\par
}

Frame 2 has no comment info
Frame 3 has no comment info
Frame 4 has no comment info
Frame 5 has no comment info....