Avoiding XPath: Part VI

Updated some 1/11/2020 -RMF

So this piece will begin a discussion about grokking Windows Security Event Logs inside rdata.table and Postgres. Use auditpol to set up kernel logging. We recall that to convert your EVTX archived security logs to CSV we need a Powershell function as below:

# Powershell Memory and CPU intensive
Function Convert-Logs3 {
[cmdletbinding()]
Param(
$filelist=$NULL
)
$filelist | foreach-object {
#Note that I am only getting four columns of data
Get-WinEvent -Path "$PSItem"| Select RecordID,ID,TimeCreated, Message | export-csv -notypeinformation -path $(write "$PSItem.csv");
[System.gc]::collect();
}
}

Convert-Logs3 "Archive-Security-2019-12-23-04-25-01-062.evtx"

#R
library(data.table)
d <- fread("Archive-Security-2019-12-23-04-25-01-062.evtx.csv")
h <- gsub('\n\t',' ',d$Message,fixed=TRUE)
h <- gsub('\n\n',' ',h,fixed=TRUE)
h <- gsub('\t\t',' ',h,fixed=TRUE)
h <- gsub('\n',' ',h,fixed=TRUE)
h <- gsub('\t',' ',h,fixed=TRUE)
h <- gsub('%%','',h,fixed=TRUE)
h <- gsub('\r','',h,fixed=TRUE)
h <- gsub('\r \r','',h,fixed=TRUE)
d$Message <- h

This produces row headings with data types:

names(d)
[1] "RecordId"    "Id"          "TimeCreated" "Message"
 
d[,sapply(.SD,class)]
   RecordId          Id TimeCreated     Message
  "integer"   "integer" "character" "character" 


# where you can use sample rdata.table queries like this:

d[,.N,.(Id,Message=substr(Message,0,100))][order(-N)]
d[Id == 4688,.(Id,TimeCreated,Message=substr(Message,275,375))]
d[Id == 4672 & !duplicated(substr(TimeCreated,0,11)),
.(Id,TimeCreated,Date=substr(TimeCreated,0,11),Message=substr(Message,400,500))]
d[Id == 4624 & grepl("S-1-0-0",Message),.(Id,Message=substr(Message,200,400))]
d[Id ==  4907,.(Id,TimeCreated,paste0(substr(Message,0,50),substr(Message,250,375)))]

There is probably much more you can do with Security.evtx files with mlr3 or rdata.table

# and then...I write out the file to be imported into Postgres:
write-csv(d, "Archive-Security-2019-12-23-04-25-01-062.csv")

Avoiding XPath : Part V

(under construction; updated 4/16/2014)

Working with such large files in R has proved to be a real challenge for 4 GB RAM.  I am studying various methods to overcome some of the challeges, but also am considering just working in Postgres or RPostgresSQL.


Once EVTX logs are in CSV format, R has the potential to parse them effectively, although I believe I am one tricky use of 'reshape'  from  doing this in the script below.

Avoiding XPath: Part IV

 Full source  of my unpolished cruft is far below.  If you are going to pull fields out of the Message in Windows Event log without Xpath or XML, how are you going to do it in Powershell 4.0? I will remind you of what the Message field looks like:


Message              : The Windows Filtering Platform has permitted a connection.

                       Application Information:
                           Process ID:        3116
                           Application Name:    \device\harddiskvolume3\users\rferrisx\appdata\local\chromium\application\chrome.exe

                       Network Information:
                           Direction:        Outbound
                           Source Address:        192.168.0.11
                           Source Port:        2094
                           Destination Address:    8.247.65.200
                           Destination Port:        80
                           Protocol:        6

                       Filter Information:
                           Filter Run-Time ID:    211332
                           Layer Name:        Connect
                           Layer Run-Time ID:    48

So I can get to these fields with 'properties':

Avoiding XPath : Part III

(under construction)

Time sequenced lattice chart of  417,834 Windows Security log entries.

This cruft is similar to my post  Avoiding XPath : Part II.  However, here I am (laboriously) converting EVTX to CSV with Powershell 3.0. The files  are sizeable, taking about an hour to convert to CSV. The charts take a non-negligible amount of time to load on my i5 4GB laptop.

Avoiding XPath : Part II

(under construction)

There are a number of issues in understanding Windows security event logging (Auditing). Every 2 - 8 weeks I collect a 614MB Security archive with just my laptop and the audit categories below.  Visualizing and manipulating this data effectively... Here are some solutions using Powershell with R 3.01.

Avoiding XPath: Part I

Typically a Security Event Log entry contains a Message entry that is resistant to anything but an XPath query:

Index         : 65597968
TimeGenerated : 1/29/2014 1:32:01 PM
EventID       : 5158
Message       : The Windows Filtering Platform has permitted a bind to a local port.

                Application Information:
                    Process ID:        200
                    Application Name:    \device\harddiskvolume3\program files (x86)\google\chrome\application\chrome.exe

                Network Information:
                    Source Address:        ::
                    Source Port:        49837
                    Protocol:        17

                Filter Information:
                    Filter Run-Time ID:    0
                    Layer Name:        %%14608
                    Layer Run-Time ID:    38

We can shelve the XPath query by using a combination of Powershell and R to dump the Message field to a single line text entry in a CSV.  Using .NET interfaces to dump the first 1000 Security log entries:

"Powershell Semantics" : The Book

Readers:

I am writing a book on Powershell. I am placing the alpha version of the book online since there is no telling when I will have the whole book written and also since this gives readers a chance to comment. If you use my material and feel the need to compensate me for such; a donation button is available on this blog.

Thanks,

Ryan M. Ferris

• As of 09/17/2013, the Introduction is available

.NET Ping and RTT object collection

[This example is code from a book I am writing to be titled "Powershell: Logic, Syntax, and Semantics".   All rights for the code below are reserved.] The code below uses .NET as the basis for a ping by name or IP function. RTT statistics are returned as objects.

# Uncomment two lines below to expose Methods and Properties:
# ([System.Net.NetworkInformation.Ping]).DeclaredMethods.Name
# ([System.Net.NetworkInformation.PingOptions]).DeclaredProperties.Name


$ping = new-object System.Net.NetworkInformation.Ping
$pingoptions = new-object System.Net.NetworkInformation.PingOptions
$pingoptions.ttl=255
$pingoptions.dontfragment=$false


function pingstring($x,$y) {$i=0;$count=$y; do {$ping.Send("$x");$i++} while($i -lt $count)}


[array[]]$hosts="nyt.com","latimes.com","sfgate.com","wapo.com"
Function Ping-Collection ($hosts, $PingCount) {
foreach ($i in $hosts) {
$host_stats = $i | foreach-object {pingstring $psitem $PingCount }
New-Object psobject -property @{
Host = [STRING]$i
IPAddresses=($host_stats.get_Address()).IPAddressToString
RTTs = $host_stats.roundtriptime
RTT_Count = [FLOAT]($host_stats | Measure-object -property roundtriptime).count
RTT_Sum = [FLOAT]($host_stats | Measure-object -property roundtriptime -sum).sum
RTT_Average = [FLOAT]($host_stats | Measure-object -property roundtriptime -average).average
} | Select @{Name="IPAddress";Expression={$_.IPAddresses | get-unique}},Host,RTTs,RTT_Count,RTT_Sum,RTT_Average
}
}


#[Result as show below]


PS C:\ps1> Ping-Collection $hosts 12 | ft -auto

IPAddress       Host        RTTs                RTT_Count RTT_Sum RTT_Average
---------       ----        ----                --------- ------- -----------
170.149.172.130 nyt.com     {18, 19, 20, 20...}        12     241    20.08333
163.192.187.17  latimes.com {79, 71, 86, 73...}        12     884    73.66666
174.143.228.153 sfgate.com  {58, 58, 57, 56...}        12     695    57.91667
208.185.109.113 wapo.com    {90, 91, 93, 91...}        12    1103    91.91666

For a class C subnet range of IP Addresses of 10 pings  of default packet size (255 addresses * 10 RTT * 32 bytes) :

$hosts = foreach($i in (1..255)) {[IPAddress]"23.1.1.$i"}

PS C:\> measure-command {$PingCollection=Ping-Collection $hosts 10}

Days              : 0
Hours             : 0
Minutes           : 7
Seconds           : 15
Milliseconds      : 619
Ticks             : 4356193908
TotalDays         : 0.00504189109722222
TotalHours        : 0.121005386333333
TotalMinutes      : 7.26032318
TotalSeconds      : 435.6193908
TotalMilliseconds : 435619.3908

PS C:\> $PingCollection.count
255
PS C:\> $PingCollection.count | sort RTT_Average | ft -auto | more
255

PS C:\> $PingCollection | sort RTT_Average | ft -auto | more

IPAddress  Host       RTTs                    RTT_Count RTT_Sum RTT_Average
---------  ----       ----                    --------- ------- -----------
           23.1.1.255 {0, 0, 0, 0...}                10       0           0
           23.1.1.1   {0, 0, 0, 0...}                10       0           0
23.1.1.5   23.1.1.5   {99, 98, 99, 100...}           10    1084       108.4
23.1.1.137 23.1.1.137 {129, 115, 124, 117...}        10    1097       109.7
23.1.1.40  23.1.1.40  {110, 136, 106, 106...}        10    1125       112.5
23.1.1.198 23.1.1.198 {111, 106, 121, 109...}        10    1131       113.1
23.1.1.27  23.1.1.27  {117, 107, 132, 108...}        10    1132       113.2
23.1.1.33  23.1.1.33  {106, 142, 110, 108...}        10    1137       113.7
23.1.1.251 23.1.1.251 {107, 119, 113, 111...}        10    1138       113.8
23.1.1.41  23.1.1.41  {133, 106, 102, 101...}        10    1146       114.6
23.1.1.178 23.1.1.178 {111, 110, 119, 134...}        10    1147       114.7

....

Nping and Powershell

[This example is code from a book I am writing to be titled "Powershell: Logic, Syntax, and Semantics".   All rights for the code below are reserved.]


I am working with how other security software works with Powershell.  This is nping 0.6.40:

sl "C:\Program Files (x86)\Nmap"
[ARRAY]$Array=foreach ($i in $e) {
$a = nping -q -H -c 10 --tcp -p 443 $i;
[ARRAY]$b = ($a[2] -replace(" ","`r`n") | findstr 'ms') -replace("ms","");
New-object  PSObject -Property @{
HOST = $i
MAX = [FLOAT]$b[0];
MIN  = [FLOAT]$b[1];
AVG  = [FLOAT]$b[2];
}
}
$Array | Select HOST,MAX,MIN,AVG | sort AVG | ft -auto


HOST                   MAX MIN     AVG
----                   --- ---     ---
rmfmedia.com           272 156 204.452
rmfnetworksecurity.com 418 168 241.996
rmfdevelopment.com     444 170 249.843

Graphing Event Logs: Muxing Powershell and R Programming

# Powershell 3.0 using .NET access to event logs
function WinEvent($EventLogName,$EL_Limit){
mkdir -ea 0 C:\RProgramming
sl -ea 0 C:\RProgramming
$computername=(gwmi -class Win32_NetworkAdapterConfiguration | % {if ($_.IPAddress -ne $null) {$input}}).PSComputerName
$a=[System.Diagnostics.EventLog]
$b=$a::GetEventLogs($computername)
$b
$b | export-csv -NoTypeInformation C:\RProgramming\EventLogs.csv
$N=((($b.Log) | sls $EventLogName).LineNumber)[0] - 1
$EventLog=$b[$N].get_Entries()[0..($b[$N].entries.count -1)]
if ($EL_Limit -eq "max") {$EL_Limit = $b[$N].entries.count -1}
$EL_fields=$Eventlog[0..$EL_Limit] | Select EventID,Index,CategoryNumber,EntryType,Source,InstanceID,TimeGenerated,TimeWritten,UserName
$EL_fields | export-csv -NoTypeInformation EventLogFile.csv
$EL_fields | group -noelement -property EventID | 
Select @{Name="EventID"; Expression ={[INT]$_.Name}}, Count | sort EventID | ft -auto
}

Examining Thread States (Part I:Wait States)

# 11:13 AM 12/13/2012 Wait-Reasons known and unknown
#  foreach ($id in ( Get-Process | ? {$_.Modules} )) {$id=($id.Threads | where {$_.WaitReason -eq "Unknown"}).ID;$id }
# 11:16 AM 12/13/2012 waiting state unknown
function thread_unknown
{
rv -ea 0 id
foreach ($i1 in ( Get-Process | ? {$_.Modules} )) {$id+=($i1.Threads | where {$_.WaitReason -eq "Unknown"}).ID}
foreach ($i2 in ( gwmi win32_thread )) {if ($i2.handle -in $id) {$i2 | Select `
@{name="ProcName";Expression={$((get-process -id $_.ProcessHandle).Name)}}, `
@{name="ProcID";Expression={$_.ProcessHandle}}, `
@{name="ProcHndleCt";Expression={$((get-process -id $_.ProcessHandle).HandleCount)}}, `
@{name="ThreadID";Expression={$_.Handle}}, `
threadstate,threadwaitreason,KernelModetime}} `
}

# 10:44 AM 12/14/2012 all thread wait reasons 
function thread_states
{
rv -ea 0 id
foreach ($i1 in ( Get-Process | ? {$_.Modules} )) {$id+=($i1.Threads | where {$_.WaitReason}).ID}
foreach ($i2 in ( gwmi win32_thread )) {if ($i2.handle -in $id) {$i2 | Select `
@{name="ProcName";Expression={$((get-process -id $_.ProcessHandle).Name)}}, `
@{name="ProcID";Expression={[INT]$_.ProcessHandle}}, `
@{name="ProcHndleCt";Expression={[INT]$((get-process -id $_.ProcessHandle).HandleCount)}}, `
@{name="ThreadID";Expression={[INT]$_.Handle}}, `
threadstate,threadwaitreason,KernelModetime}} `
}

$thread_states=thread_states
$thread_states | group -property KernelModetime | Select Count, @{Name="KernelModeTime";Expression={[INT]$_.Name}},Group | sort -desc KernelModeTime | ft -auto

Count KernelModeTime Group
----- -------------- -----
    1         277416 {@{ProcName=NServiceEntry; ProcID=1772; ProcHndleCt=111; ThreadID=1864; threadstate=5; threadwaitreason=4; KernelModetime=277416}}
    1         274702 {@{ProcName=MsMpEng; ProcID=944; ProcHndleCt=468; ThreadID=1828; threadstate=5; threadwaitreason=15; KernelModetime=274702}}
    1         102617 {@{ProcName=chrome; ProcID=1412; ProcHndleCt=3261; ThreadID=2020; threadstate=5; threadwaitreason=6; KernelModetime=102617}}
    1          52431 {@{ProcName=chrome; ProcID=1412; ProcHndleCt=3261; ThreadID=5564; threadstate=5; threadwaitreason=15; KernelModetime=52431}}
...

 $thread_states | ? {$_.threadwaitreason -eq "21"} |  group -property KernelModetime | Select Count, @{Name="KernelModeTime";Expression={[INT]$_.Name}},Group | sort -desc KernelModeTime | ft -auto


Count KernelModeTime Group
----- -------------- -----
    1            842 {@{ProcName=chrome; ProcID=1412; ProcHndleCt=3261; ThreadID=5588; threadstate=5; threadwaitreason=21; KernelModetime=842}}
    1            624 {@{ProcName=chrome; ProcID=1412; ProcHndleCt=3261; ThreadID=5236; threadstate=5; threadwaitreason=21; KernelModetime=624}}
    1            436 {@{ProcName=chrome; ProcID=1412; ProcHndleCt=3261; ThreadID=4272; threadstate=5; threadwaitreason=21; KernelModetime=436}}
    1            218 {@{ProcName=chrome; ProcID=1412; ProcHndleCt=3261; ThreadID=5392; threadstate=5; threadwaitreason=21; KernelModetime=218}}
    1              0 {@{ProcName=svchost; ProcID=1544; ProcHndleCt=371; ThreadID=1560; threadstate=5; threadwaitreason=21; KernelModetime=0}}

thread_unknown | ft -auto

ProcName ProcID ProcHndleCt ThreadHandle threadstate threadwaitreason KernelModetime
-------- ------ ----------- ------------ ----------- ---------------- --------------
svchost  1544           369 1560                   5               21              0
chrome   1412          3270 5588                   5               21            842
chrome   1412          3270 4272                   5               21            436
chrome   1412          3270 5236                   5               21            624
chrome   1412          3270 5392                   5               21            218

Less Thrashing; Better Queries (Part V)

# Updated: 7:01 AM 1/19/2014  -RMF

# Using [System.Diagnostics.EventLog] for Powershell 3.0 Beta
# Code
# Clearing variable types 
rv -ea 0 i;
rv -ea 0 var
$var=@("a","b","c","d","e"); foreach ($i in $var) {rv -ea 0 $i}
 #Creating $a specific to the 'GetEventLogs()' 
 # method for [System.Diagnostics.EventLog]
 $a=[System.Diagnostics.EventLog]::GetEventLogs()
 $a | gm -s
 # List the event logs
 $a

 # Creating $a as generic to the .NET class; Querying active
 # Eventlog for a local(or remote?)computer name:
 $a=[System.Diagnostics.EventLog]
 $a::GetEventLogs("rmfvpc")
 $a::GetEventLogs("rmfvpc") | gm -s

 # Creating $B as the result of mahine specific
 #'GetEventLogs()' query
 $b=$a::GetEventLogs("rmfvpc")
 $b | gm -s
 $b | gm -f

 # Using $B to get a specific method for a specific log (e.g. 
 # Array[10]) for specific configuration method (e.g. 
 # 'get_OverflowAction()')
 $b[10]
 $b[10].get_OverflowAction()

 # Choose the Security Log
 $C = $B | Where Log -eq Security

 # This retrieves all Entries before returning the first index.
 $c[0].get_Entries()[0]
 $c[0].get_Entries()[0] | gm -f

 #Returns select entries and then select EventIDs for such.
 $c[0].get_Entries()[100..110]
 $c[0].get_Entries()[100..110]
 $c[0].get_Entries()[100..110].get_EventID()

 # Number of Events Logs; Number of total events for a 
 # specific Event Log.
 $c[0].count
 $c[0].Entries.count

 # Returns First and Last Events
 $lc = $($c[0].Entries.count - 1) 
 $c[0].get_Entries()[0,$($c[0].Entries.count - $lc)]

 # Creates a DateTime variable;Returns number of days 
 # between first and last events
 ($c[0].get_Entries()[0,$lc]).TimeGenerated
 $TG=($c[0].get_Entries()[0,$lc]).TimeGenerated
 $TG  | gm -s
 $TG[1]-$TG[0]
 ($TG[1]-$TG[0]).Days

 # Returns select sorted information last 1000 entries
 $d=($c[0].get_Entries())[0..999]| Select EventID,Message
 $d.count
 $d[0..9] | ft -auto -wrap
 $d | group -property EventID -noelement | sort -desc -property Count
 $e= ($d | group -property Message -noelement | sort -desc -property Count)
 $e.count
 $e[0..9] | ft -auto -wrap

Less Thrashing; Better Queries (Part IV)

I will just keep running with this series until I get sick of [System.Diagnostics.Eventing...].

The code below  compares an exported list of  "Security Audit Events for Windows 7 and Windows Server 2008 R2" with a grouped list of the last 10K events from the Security Log. This is a quick way to look at all your security events ordered by count and an appended "Message Summary".


# Working 3:57 PM 9/8/2012
# PS 3.0 Beta

$Security= get-winevent -ea 0  -LogName Security -MaxEvents 10000
$a=$Security | group -property ID -noelement | sort -desc -property count
[array[]]$b=import-csv .\Windows7.csv
[array[]]$c=0..((($B."Event ID").count) - 1) | % -process {
 foreach ($ID in $a.name) {if ($ID -eq $B[$PSItem]."Event ID") `
  {write "$($B[$PSItem]."Event ID") : $($B[$PSItem]."Message Summary")"}}}
$a
$c

PS C:\ps1> $Security=get-winevent -max 10000 Security
PS C:\ps1> $a=$Security | group -property ID -noelement | sort -desc -property count
PS C:\ps1> [array[]]$b=import-csv .\Windows7.csv
PS C:\ps1> [array[]]$c=0..((($B."Event ID").count) - 1) | % -process {
>> foreach ($ID in $a.name) {if ($ID -eq $B[$PSItem]."Event ID") `
>> {write "$($B[$PSItem]."Event ID") : $($B[$PSItem]."Message Summary")"}}}
>> $a
>> $c
>>

Count Name
----- ----
 7683 5156
 2125 5158
  121 4688
   45 5157
    8 4624
    7 4672
    3 5154
    2 4776
    2 4634
    1 4801
    1 4800
    1 4625
    1 4648
4624 : An account was successfully logged on.
4625 : An account failed to log on.
4634 : An account was logged off.
4648 : A logon was attempted using explicit credentials.
4672 : Special privileges assigned to new logon.
4688 : A new process has been created.
4776 : The domain controller attempted to validate the credentials for an account.
4800 : The workstation was locked.
4801 : The workstation was unlocked.
5154 : The Windows Filtering Platform has permitted an application or service to listen on a port for incoming connections.
5156 : The Windows Filtering Platform has allowed a connection.
5157 : The Windows Filtering Platform has blocked a connection.
5158 : The Windows Filtering Platform has permitted a bind to a local port.

Less Thrashing; More Sorting Queries (Part IIl)

The cruft below demonstrates (somewhat) how to effectively interrogate  70k events from Windows  with PS 3.0. It presumes you are using 'auditpol' to your advantage. When querying that many events, I keep a check on memory in the title bar with this function:


function Global:set-titleMemoryStats {

# With 3.0 Runspace
$set_title=

{

function Global:Set-title {
$PSID=([System.Diagnostics.Process]::GetCurrentProcess()).Id
$MemStats=ps -id $PSID | Select `
@{Name='ThreadCount';Expression={($_.Threads).count}}, `
@{Name='WorkSetMB';Expression={[int](($_.WorkingSet64)/1MB)}}, `
@{Name='VirMemMB';Expression={[int](($_.VirtualMemorySize64)/1MB)}}, `
@{Name='PriMemMB';Expression={[int](($_.PrivateMemorySize64)/1MB)}}, `
@{Name='PagedMemMB';Expression={[int](($_.PagedMemorySize64)/1MB)}}, `
@{Name='NonPagedMemKB';Expression={[int](($_.NonPagedSystemMemorySize64)/1KB)}}

$Title=write "Last_Title_Stats: Time: $([datetime]::now) Version: $((get-host).Version.Major) SessionHours: $([int]([datetime]::now - (ps -id $psid).Starttime).totalhours) Memory: $($Memstats) GC_MB: $([int]([GC]::gettotalmemory(1)/1MB))"
[console]::set_title($Title)
}
while(1) {set-title;sleep -s 5}
}

$ST_Runspace = [PowerShell]::Create().AddScript($set_title)
$Begin_Set_Title = $ST_Runspace.BeginInvoke()

# To stop all of this...
# $ST_Runspace.runspace
# $Stop_Set_Title = $ST_Runspace.Stop()
# $Dispose_Set_Title = $ST_Runspace.Dispose()
}

Less Thrashing; More Queries (Part II)

These are more advanced event queries than my last post. Creating queries as psobject properties improves query speeds.More intra-message searches as calculated properties are included below.

$Elements=New-Object PSObject -Property @{
Security_515X=get-winevent -ea 0 -max 50000 -filterhashtable @{logname='Security';ID=@(5156..5158)}
System=get-winevent -ea 0 -max 10000 -filterhashtable @{logname='System'}
Application=get-winevent -ea 0 -max 10000 -filterhashtable @{logname='Application'}
}

if ($All_Events) {rv -ea 0 All_Events}
$global:Events= foreach ($element in $Elements) {$element}
[array]$HAElements="Security_515X","System","Application"
foreach ($element in $HAElements){$All_Events+=$Events.$element}
$a=$All_Events | Sort -desc -property TimeCreated

Thrashing memory with queries...(Part I)

These are some event log search queries. They all trash my memory in any Powershell session.
I have written them down for some reason. Probably so I can evolve more elegant solutions to such queries in the future.

foreach ($i in @("a","b","c","d","e")) {if ($i) {rv -ea 0 $i}}
$start=(get-winevent  -log security -max 1).TimeCreated
$a=(get-winevent -ea 0 -max 50000 -filterhashtable @{logname='Security';ID=@(5156..5158)})
$c=(get-winevent -ea 0 -log System -max 10000) | Select TimeCreated,ProcessID,ThreadID,Message
$d=(get-winevent -ea 0 -log Application -max 10000) | Select TimeCreated,ProviderName,Message
$b=$a |  Select TimeCreated,ProcessID,ThreadID, `
@{Name="Port";Expression={($_.Message | findstr /C:"Source Port:").replace("Source Port:"," ")}}, ` 
@{Name="Application";Expression={($_.Message | findstr /C:"Application Name:").replace("Application Name:"," ")}}
$e=@()
$e=($b + $c + $d) | sort -desc -property TimeCreated
$a.count;$b.count;$c.count;$d.count;$e.count
$e | ft -auto -wrap TimeCreated,Application,ProcessID,ThreadID,Port,Message

Charting Procmon network output with .NET 4.0 and Powershell

Lots to work out in this post. Powershell v 3.0 CTP2 or Beta.  Procmon is Mark Russinovich's flagship tool for diagnosing Windows activity. It normally runs from the (admin) command prompt:

procmon /noconnect /nofilter /minimized /quiet

From Powershell admin prompt you can run thus:

start-process .\procmon.exe -arg '/LoadConfig JustNetwork.pmc' /quiet -verb runas -window hidden

whereupon a hidden procmon would run in the background capturing network traffic provided  that you have exported the configuration 'JustNetwok.pmc' to your path. You can create this filter and  export this configuration from the file menu:

Charting ordered Hash Data from the Security Event Log

Get-Winevent Part IV: Querying the Event Log for 'Filtering Platform Connection' Information (Part A)

The command:

'auditpol /set /subcategory:"Filtering Platform Connection" /success:enable /failure:enable'

enables the "Filtering Platform Connection" security counter on Windows 7. The "Filtering Platform Connection" gives your event logs access to the following counters:

Filtering Platform Connection           Success and Failure

• Object Access Filtering Platform Connection 5150 The Windows Filtering Platform has blocked a packet. Windows 7, Windows Server 2008 R2
• Object Access Filtering Platform Connection 5151 A more restrictive Windows Filtering Platform filter has blocked a packet. Windows 7, Windows Server 2008 R2
• Object Access Filtering Platform Packet Drop 5152 The Windows Filtering Platform blocked a packet. Windows Vista, Windows Server 2008
• Object Access Filtering Platform Packet Drop 5153 A more restrictive Windows Filtering Platform filter has blocked a packet. Windows Vista, Windows Server 2008
• Object Access Filtering Platform Connection 5154 The Windows Filtering Platform has permitted an application or service to listen on a port for incoming connections. Windows Vista, Windows Server 2008
• Object Access Filtering Platform Connection 5155 The Windows Filtering Platform has blocked an application or service from listening on a port for incoming connections. Windows Vista, Windows Server 2008
• Object Access Filtering Platform Connection 5156 The Windows Filtering Platform has allowed a connection. Windows Vista, Windows Server 2008
• Object Access Filtering Platform Connection 5157 The Windows Filtering Platform has blocked a connection. Windows Vista, Windows Server 2008
• Object Access Filtering Platform Connection 5158 The Windows Filtering Platform has permitted a bind to a local port. Windows Vista, Windows Server 2008
• Object Access Filtering Platform Connection 5159 The Windows Filtering Platform has blocked a bind to a local port. Windows Vista, Windows Server 2008

This script, which uses some Powershell 3.0 features, produces the output far below (abbreviated) by parsing the output from EventID 5156 ("allowed connection"). The loops are structured to allow 'findstr' to dig out 'subfield' information. 'Select -unique' functions to find unique addresses (or ports):

[array]$a=Get-WinEvent -FilterHashTable @{LogName='Security';ID=5156;StartTime=$StartTime}
$UDA_count=$a.count
[array[]]$b=$a.Message | findstr 'Destination' | findstr 'Address'
$Global:UDestAddress=($b | Select -unique) | sort

The script takes an extremely long time to run on my five core laptop. These scripts (1,2) are optimized a bit more to search for only 5156 Events. The global variables in the script would be suitable for parsing against lists of allowed ports, allowed or blocked IPs. The Script can be used as a format for other counters as well. Several features from Powershell 3.0 are used in this script including the ability of Powershell 3.0 to 'automatically unroll' an entire array for a certain property (e.g. '[array[]]$b=$a.Message'). I could dearly use a much faster Powershell method to dig 'subfield' information out of the Message field than double piping that information to 'findstr'. The issue is that a single day of network activity generates ten of thousands of kernel security counters.  An alternative to limit the amount of information returned might be to use the '-max' [number of events] parameter:

Evtsys (actually auditpol and auditusr) Part II

# Powershell V3 CTP2
# Using auditpol on Vista, Win7
# Enables failure and sucess auditing for selected subcategories

$auditpollist=
"Logon",
"Logoff",
"Special Logon",
"Other Logon/Logoff Events",
"Security State Change",
"SAM",
"Filtering Platform Connection",
"Process Creation",
"Audit Policy Change",
"Filtering Platform Policy Change",
"Credential Validation"

foreach ($i in $auditpollist) {auditpol /set /subcategory:"$i" /success:enable /failure:enable}



# Using auditusr on XP, 2003
# Since auditusr requires doesn't globally audit all users...

$auditusrlist=
"System Event",
"Logon/Logoff",
"Object Access",
"Privilege Use",
"Detailed Tracking",
"Policy Change",
"Account Management",
"Account Logon"

# creates list of all XP users
function netusers {$query = "Win32_UserAccount";$query+= " WHERE LocalAccount='True'";Get-WmiObject $query }
$name_list=(netusers)
[array]$name_list=foreach ($i in $name_list) {$i.name}

# set success and failure for all users for all categories in $auditusrlist
$name_list | % -process {
foreach ($i in $auditusrlist) {$au_str="$_`:$i";auditusr /is $au_str};
foreach ($i in $auditusrlist) {$au_str="$_`:$i";auditusr /if $au_str};
}