Get-WinEvent, EventLogs, ETL, Providers on Win7 Part II

Working with Windows Tracing (ETL) logs
This is part of ongoing research project to understand how improved tracing providers in Windows 7 can help detect the presence of malware.  Microsoft has been improving event tracing for a number of years. The latest versions allows netsh to invoke multiple providers. After you have chosen your providers, you start the trace either by referencing the provider name or GUID. 'Netsh trace start' allows for keyword or capture filters, which can be useful if you know what specific events for which you need to trace. For this example, we will not create an NDIS capture ('capture=yes') nor will we select keywords or levels for the filters. After a few busy hours, this leads to quite a bit tracing.

Get-WinEvent, EventLogs, ETL, Providers on Win7

'Get-WinEvent' in Powerhsell 2 when combined with ETL on Windows 7 allows exceptional event log queries. This function allows the administrator to create an array of all Event Logs and sort by 'time created' all those records created in the last (1) day:



function global:LatestLogEntries
{
   [CmdletBinding()]
   Param(
       [Parameter(Mandatory=$true,ValueFromPipeline=$true)]
       [int32] $param1,
       [string] $ErrorActionPreference="silentlycontinue"
            )


$LogNames=(Get-Winevent -listlog  * )
$goback = (get-date) - (new-timespan -days $param1 )
$LogNames | % {get-winevent -FilterHashTable  @{LogName=$_.LogName;StartTime=$goback}}
}

Powershell LSOF/Parsing Netstat Part II

Two 'lsof for Powershell' scripts covering v4 and v6 have been placed here:

hhttp://rmfdevelopment.com/PowerShell_Scripts/PS_LSOF.ps1
http://rmfdevelopment.com/PowerShell_Scripts/PS_LSOF_gwmi.ps1

This is a second update to this script which matches the port to the process in Powershell by parsing netstat for TCP and UDP and then appending 'ps' or 'gwmi' information associated with the process related to that port.  There's nothing in this function (but sorted port order) which carries through a relational tie from port to process information. There is a lot of information produced in this script, as I print all of netstat -ano and then query the corresponding network process with either 'ps' or 'gwmi'. (Click to enlarge):

Powershell LSOF / Parsing Netstat

Update 09/14/2012:

Other attempts at an lsof for Windows are here:

• thinking-about-network-security.blogspot.com/2010/10/powershell-lsof-parsing-netstat.html
• thinking-about-network-security.blogspot.com/2010/12/powershell-lsofparsing-netstat-part-ii.html
• thinking-about-network-security.blogspot.com/2010/05/lsof-for-windows-subsitute.html

These are very 1.0 and 2.0.  I will try to update my lsof attempts to 3.0 soon.

-RMF



This script, parse-netstat.ps1, successfully parses 'netstat -ano' for each PROTO (TCP,TCPv6,UDP, UDPv6) and then uses 'ps' to enumerate ID,NAME,PATH,FileVersion for the process associated with each networked PID. Thus we have a basic Powershell LSOF utility with room for calculated properties and additional text parsing. There is no spec of regex anywhere in my text parsing of netstat.  Sample output:


PS C:\ps1> .\parse-netstat.ps1
TCP Local Ports:
135
445
1025
1026
1027
1028
1031
9000
24800
47001
139
24800
139
1095
1099
1100
1101
1102
1679
1706
TCP PIDS:


  Id Name     Path                                                              FileVersion
  -- ----     ----                                                              -----------
1012 svchost  C:\Windows\system32\svchost.exe                                   6.0.6000.16386 (vista_rtm.061101-2205)
   4 System
 684 wininit  C:\Windows\system32\wininit.exe                                   6.0.6000.16386 (vista_rtm.061101-2205)
 460 svchost  C:\Windows\System32\svchost.exe                                   6.0.6000.16386 (vista_rtm.061101-2205)
 760 lsass    C:\Windows\system32\lsass.exe                                     6.0.6000.16386 (vista_rtm.061101-2205)
  12 svchost  C:\Windows\system32\svchost.exe                                   6.0.6000.16386 (vista_rtm.061101-2205)
 740 services C:\Windows\system32\services.exe                                  6.0.6000.16386 (vista_rtm.061101-2205)
   4 System
4244 synergys C:\Program Files (x86)\Synergy+\bin\synergys.exe
   4 System
   4 System
4244 synergys C:\Program Files (x86)\Synergy+\bin\synergys.exe
   4 System
 552 Picasa3  C:\Program Files (x86)\Google\Picasa3\Picasa3.exe                 3.6.105.67
 552 Picasa3  C:\Program Files (x86)\Google\Picasa3\Picasa3.exe                 3.6.105.67
 552 Picasa3  C:\Program Files (x86)\Google\Picasa3\Picasa3.exe                 3.6.105.67
 552 Picasa3  C:\Program Files (x86)\Google\Picasa3\Picasa3.exe                 3.6.105.67
 552 Picasa3  C:\Program Files (x86)\Google\Picasa3\Picasa3.exe                 3.6.105.67
   4 System
4460 chrome   C:\Users\Admin\AppData\Local\Google\Chrome\Application\chrome.exe 0.0.0.0

Accessing (or not) GetOwnerModuleFromTcpEntry from Powershell

Normally on XP SP2, Vista, Win7 'netstat -ano' or 'netstat -anob' gives us the connected sockets, the PID of listening applications. With the '-b' option, netstat makes an attempt at finding the owner of the socket probably through the 'GetOwnerModuleFromTcpEntry function [which] retrieves data about the module that issued the context bind for a specific IPv4 TCP endpoint in a MIB table row.'  found in iphlpapi.dll (IP Helper). Finding this same information with Powershell I have found to be more than difficult. It is easy enough to find the listening and connected sockets with [System.NET.NetworkInformation.IPGlobalProperties]::GetIPGlobalProperties(). 

Check-TCPUDPClient.ps1

The output from the script below is designed to be a framework to check TCP and UDP open ports under connection. It makes use of whatever TCP and UDP Client sockets code is native to Powershell 2.0. My original conception was to create a scripted 'fuzzer' that would send non-arbitrary data to open ports to test or provoke library module loading.  Powershell's socket facilities are impressive for a scripted language. I don't know how much documentation there is for TCP/IP.  No error checking implemented.

Check-TCPUDPClient.ps1

Looking at Process, Threads, Modules with Powershell 2.0

I have published "Looking at Processes, Modules, and Threads with Powershell 2.0 Part I".  The paper concerns itself with comparing Processes, Modules, and Threads and offers some discussion for comparing their changes over time. See also:
http://www.rmfdevelopment.com/PowerShell_Scripts/diff_PMT.ps1
http://rmfdevelopment.com/PowerShell_Scripts/diff_PMT_adv.ps1

Argus!!!

I have been reading Real Digital Forensics and came across the recommended use of Argus ("Audit Record Generation and Utilization System"). Argus is fast, wide and deep network analysis of pcap files.  It took me some time to compile and start to make sense of it, although there is a relevant and clever wiki page and a good collection of recent articles explaining research, university and real world use. My discussion below concerns Argus auditing functionality.

Argus dumps your pcap file into a compressed argus formatted file which carries every piece of session information an inquisitive NSM forensic could possibly want from a network trace including time-slices, TCP options, anonymization, geolocation, and graphing . Here are some basic examples I walked myself through. The first step is to write the pcap file to an argus file using 'argus'.

/usr/local/sbin/argus -d -r 08Mar1142PST2010.in.1268074842 -w 08Mar1142PST2010.in.1268074842.argus

Next I use 'ra' (read argus)  to read the packet data.  You can specify fields and bpf style filters. Here I specify (append) a filter ('ip proto 6') for only TCP packets  (e.g grep TCP /etc/protocols):
  
ra -n -r 08Mar1142PST2010.in.1268074842.argus - ip proto 6 | less
19:08:09.660222 e s tcp 207.44.254.106.56813 -> 192.168.0.12.3246 3 186 REQ
19:12:01.707471 e tcp 204.236.155.168.12200 -> 192.168.0.12.3246 1 60 REQ
19:32:55.259094 e tcp 204.236.155.168.12200 -> 192.168.0.12.3246 1 60 REQ
19:33:44.995964 e tcp 221.192.199.35.12200 -> 192.168.0.12.8000 1 60 REQ
19:34:36.506022 e tcp 221.192.199.35.12200 -> 192.168.0.12.80 1 60 REQ
19:53:52.914418 e tcp 204.236.155.168.12200 -> 192.168.0.12.3246 1 60 REQ

Here I specify source address, destination port and connection state fields with the '-s' option and sort the result by source address and destination port before using 'uniq -c' to rank those fields.

ra -n -s saddr dport state -r 08Mar1142PST2010.in.1268074842.argus - ip proto 6 | sort -k1,2 -nr | uniq -c | sort -nr | less
149 221.195.73.86 8000 REQ
100 192.168.0.12 80 ACC
81 222.45.112.59 2479 REQ
80 222.45.112.59 8085 REQ
80 222.45.112.59 3246 REQ
76 204.236.155.168 3246 REQ

I am using 'rasort' to something similar here but appending grep to filter only those source addresses with a connected state.

 rasort -n -s saddr dport state -r 08Mar1142PST2010.in.1268074842.argus - ip proto 6 | sort -k1 -nr | uniq -c | sort -nr | grep CON | less
14 74.125.19.19 19412 CON
14 74.125.19.17 20073 CON
13 85.13.200.108 19216 CON
13 85.13.200.108 19024 CON
13 74.125.19.83 19145 CON
13 74.125.19.83 18961 CON

I am not quite clear when to use 'rasort'  versus 'ra' with sort and uniq appended.  There is also 'ratop' . May take some time to sort out the best scripts for top talkers. Like 'ra', I can tell 'rasort' to include specific field (-s switch) and then specify  the field(s) to sort by (-m  switch). I am still using 'uniq -c | sort -r' .

rasort -s saddr dport proto bytes stat -m dport saddr  -r 08Mar1142PST2010.in.1268074842.argus | grep -v -f file | uniq -c | sort -r | less

149 221.195.73.86 8000 tcp 60 REQ
81 222.45.112.59 2479 tcp 60 REQ
80 222.45.112.59 8085 tcp 60 REQ
80 222.45.112.59 3246 tcp 60 REQ
76 204.236.155.168 3246 tcp 60 REQ
76 222.45.112.59 9415 tcp 60 REQ


So here I apply a bpf filter for dst port 22 and the '-z' to see TCPstate changes :
  
rasort -nn -s saddr dport proto bytes state -m dport saddr -z -r 08Mar1142PST2010.in.1268074842.argus - dst port 22 | uniq -c | sort -nr

3 125.141.195.190 22 6 62 s
3 114.202.247.235 22 6 62 s
3 58.217.255.103 22 6 62 s
3 97.163.189.33 22 6 62 s
2 94.158.184.183 22 6 62 s
2 61.151.246.140 22 6 62 s
 
Argus, baby!! Fast, wide and deep!!

the 'find' command for security...Part I

These are some meditations on using the *NIX 'find' command for security...

time stamping windows directory and file names

This is something I have blogged about before, but I thought it worth posting again.  Special characters need to be eliminated to create a time stamp that can be used as a Windows file name. The `date` program in Unix has a number of very useful options for this.  Windows cmd shell is more limited. This is what I use:

:: rtime.cmd
@echo off

set realdate=%date:/=.%
set realdate=%realdate:* =%
set realtime=%time::=.%
set realtime=%realtime:* =%
set timestamp=%realdate%.%realtime%
echo %timestamp%

This command script uses 'variable substitution' from the set command to remove special characters (e.g. :  / ) unacceptable as Windows file or directory names . This line:
set timestamp=%realdate%.%realtime%


can be changed as needed for more CSV compatible logging:
set timestamp="%realdate%","%realtime%"


Once cached, it runs pretty fast and is suitable for lightweight logging:

$ time /cygdrive/C/Security/rtime.cmd
06.02.2010.11.04.05.99

real    0m0.202s
user    0m0.015s
sys     0m0.031s

$ time /cygdrive/C/Security/rtime.cmd
06.02.2010.11.04.12.65

real    0m0.062s
user    0m0.000s
sys     0m0.015s

$ time /cygdrive/C/Security/rtime.cmd
06.02.2010.11.04.14.68

real    0m0.062s
user    0m0.000s
sys     0m0.015s

piping tcpdump output to lsof

This simple Bash script will output the lsof end of any foreign network connection:
[Set to the interface of your choice]
while [ 1 ]
        do
                for i in `tcpdump -i rl0 -c 1 -l dst $(hostname) | awk '{print $2}' | awk -F"." '{print $1"."$2"."$3"."$4}'`
                         do lsof -i@$i
                done
done
with time/date stamp added and headers removed:
while [ 1 ]

        do
                for i in `tcpdump -i rl0 -c 1 -l dst $(hostname) | awk '{print $2}' | awk -F"." '{print $1"."$2"."$3"."$4}'`
                         do echo `date -u` `lsof -i@$i | grep -v PID`
                done
done

Run like this:
./tcp_lsof.sh >> tcp.lsof.log &

the script produces output like this:

COMMAND   PID     USER   FD   TYPE     DEVICE SIZE/OFF NODE NAME
sshd    18392 rferrisx    5u  IPv4 0xd699ac80      0t0  TCP rmflaptop.rmfdevelopment.com:ssh->192.168.0.3:13974 (ESTABLISHED)
sshd    29850     root    5u  IPv4 0xd699ac80      0t0  TCP rmflaptop.rmfdevelopment.com:ssh->192.168.0.3:13974 (ESTABLISHED)
or
Wed May 26 15:22:06 UTC 2010 sshd 9448 root 5u IPv4 0xd699ac80 0t0 TCP rmflaptop.rmfdevelopment.com:ssh->192.168.0.3:15729 (ESTABLISHED) 
sshd 29734 rferrisx 5u IPv4 0xd699ac80 0t0 TCP rmflaptop.rmfdevelopment.com:ssh->192.168.0.3:15729 (ESTABLISHED)
Wed May 26 15:22:07 UTC 2010 sshd 9448 root 5u IPv4 0xd699ac80 0t0 TCP rmflaptop.rmfdevelopment.com:ssh->192.168.0.3:15729 (ESTABLISHED) 
sshd 29734 rferrisx 5u IPv4 0xd699ac80

A prototype test harness...but needs lots of work

I have spent too much time here in the last few days working on a test harness for live network files in Vista. As a prototype, what I have written may be useful. However, numerous problems were uncovered.  The idea was this: At any moment they are a discoverable set of files that are being accessed by the network. In theory, you should be able to list those files and then query them for their integrity. The heart of this is something like:  
  
icacls %dir_file%                                                                         &( 
if /I [%filetype% EQU [regular sfc /verifyfile=%dir_file% ) &(
if /I [%filetype% EQU [regular accesschk -qv %dir_file% ) &(
if /I [%filetype% EQU [regular sigcheck -q %dir_file% )

Definitely some useful information is returned. But the project will have to be rewritten in a faster language with better string support. Interesting to see what information it did return. Like the file - C:\Windows\System32\nsi.dll - below.

 Running icacls, sfc, accesschk, sigcheck for FileType,FileID,Path: regular 1220: "C:\Windows\System32\nsi.dll "
filetype=regular
C:\Windows\System32\nsi.dll NT SERVICE\TrustedInstaller:(F)
BUILTIN\Administrators:(RX)
NT AUTHORITY\SYSTEM:(RX)
BUILTIN\Users:(RX)

Successfully processed 1 files; Failed processing 0 files

Windows Resource Protection could not perform the requested operation.
C:\Windows\System32\nsi.dll
Medium Mandatory Level (Default) [No-Write-Up]
RW NT SERVICE\TrustedInstaller
FILE_ALL_ACCESS
R BUILTIN\Administrators
FILE_EXECUTE
FILE_LIST_DIRECTORY
FILE_READ_ATTRIBUTES
FILE_READ_DATA
FILE_READ_EA
FILE_TRAVERSE
SYNCHRONIZE
READ_CONTROL
R NT AUTHORITY\SYSTEM
FILE_EXECUTE
FILE_LIST_DIRECTORY
FILE_READ_ATTRIBUTES
FILE_READ_DATA
FILE_READ_EA
FILE_TRAVERSE
SYNCHRONIZE
READ_CONTROL
R BUILTIN\Users
FILE_EXECUTE
FILE_LIST_DIRECTORY
FILE_READ_ATTRIBUTES
FILE_READ_DATA
FILE_READ_EA
FILE_TRAVERSE
SYNCHRONIZE
READ_CONTROL
c:\windows\system32\nsi.dll:
Verified: Signed
Signing date: 8:08 AM 1/19/2008
Strong Name: Unsigned
Publisher: Microsoft Corporation
Description: NSI User-mode interface DLL
Product: Microsoft« Windows« Operating System
Version: 6.0.6001.18000
File version: 6.0.6001.18000 (longhorn_rtm.080118-1840)

Car hacking....

"Indeed, we have demonstrated the ability to systematically control a wide array of components including engine, brakes, heating and cooling, lights, instrument panel, radio, locks, and so on. Combining these we have been able to mount attacks that represent potentially significant threats to personal safety. For example, we are able to forcibly and completely disengage the brakes while driving, making it difficult for the driver to stop. Conversely, we are able to forcibly activate the brakes, lurching the driver forward and causing the car to stop suddenly."

Great Reading! Great Research!

How would you like to pwn your first car?

lsof for Windows subsitute

5/10/2010 update to this post (see below)
I've created a couple of Vista cmd files that pump netstat output to tasklist to help substitute for the missing`lsof -Ts` in Linux (see below). The TCP/TCPv6 output logs the time, IP address (foreign endpoint), application information. The (stateless) UDP/UDPv6 output just logs time and application information.  (See output below). The value of logging network endpoints and their process information is incalculable in security. Mark Russinovich's procmon (when run with the network filter) does this quite thoroughly.  Microsoft's Netmon 3.3 correlates endpoint data packets to most applications. However, I was interesting in developing something cmd line, perhaps not so heavy, using all native Vista commands. The crux of the scripts are:

:: pipe appropriate netstat output to tasklist
for /f "tokens=1-5" %%a in ('@netstat -%netstat_option% -p %connection_type% ^| findstr /V Active ^| findstr /V Proto') do set EP=%%c& set PID=%%e& call :loop
..
::log Endpoint and network process PID
@echo "%timestamp%","%EP%", | findstr /V "ECHO"
@tasklist /FO CSV /V /FI "PID eq %PID%" /NH

 With an automated check of network %PID% in place, you can add options to check/log the open files of each network application with the (very slow) 'openfiles' command:
[The 'openfiles' cmd works once global flags are enabled.]

for /f "tokens=1-5" %a in ('openfiles /query /FO TABLE /NH /V') do @if %c==%PID% echo %e >> temp
...
C:\Users\Admin\AppData\Local\Google\Chrome\Application\4.1.249.1064
...
Adding Mark Russinovich's accesschk will show the security permissions on those files:

for /f %i in ('more temp') do @accesschk -qv %i | more
....
C:\Users\Admin\AppData\Local\Google\Chrome\Application\4.1.249.1064\avcodec-52.dll
Medium Mandatory Level (Default) [No-Write-Up]
RW RMFVista\Admin
FILE_ALL_ACCESS
RW NT AUTHORITY\SYSTEM
FILE_ALL_ACCESS
RW BUILTIN\Administrators
FILE_ALL_ACCESS
....
The cmd files can be found here: 
http://www.rmfdevelopment.com/PowerShell_Scripts/ano_TCP.cmd
http://www.rmfdevelopment.com/PowerShell_Scripts/ano_UDP.cmd


5/10/2010 update:
An update which takes any of four arguments (TCP,TCPv6,UDP, UDPv6) and logs to a CSV file output as below can be found at http://www.rmfdevelopment.com/PowerShell_Scripts/ano_all.cmd
ano_all.cmd output for TCP
"05.10.2010_11.35.21.34","LISTENING","0.0.0.0:1029","0.0.0.0:0","services.exe","740","Services","0","9,532"

"05.10.2010_11.35.21.82","LISTENING","0.0.0.0:9000","0.0.0.0:0","System","4","Services","0","21,204"

"05.10.2010_11.35.22.34","LISTENING","192.168.0.3:139","0.0.0.0:0","System","4","Services","0","21,204"

"05.10.2010_11.35.22.84","CLOSE_WAIT","192.168.0.3:1059","174.133.89.198:80","pctsSvc.exe","856","Services","0","195,660"

"05.10.2010_11.35.23.33","ESTABLISHED","192.168.0.3:1072","85.13.200.108:21","ftp.exe","2568","Console","1","6,388"

"05.10.2010_11.35.23.82","ESTABLISHED","192.168.0.3:1080","74.125.155.139:80","chrome.exe","4404","Console","1","62,576"

"05.10.2010_11.35.24.31","ESTABLISHED","192.168.0.3:1082","72.14.213.191:80","chrome.exe","4404","Console","1","62,576"

ano_TCP.cmd output for TCP
(note: It would be trivial to add the connection state as well. I did in ano_all.cmd as remarked above -RMF)

"05.06.2010_21.30.31.74","174.133.89.198:80",
"pctsSvc.exe","3368","Services","0","24,588 K","Unknown","NT AUTHORITY\SYSTEM","0:15:51","N/A"
"05.06.2010_21.30.32.20","72.14.213.99:80",
"Picasa3.exe","4248","Console","1","128,588 K","Running","RMFVista\Admin","0:02:16","Picasa 3"
"05.06.2010_21.30.32.69","72.14.213.101:80",
"chrome.exe","4232","Console","1","79,432 K","Running","RMFVista\Admin","0:01:49","Network Security - Google Chrome"
"05.06.2010_21.30.33.15","74.125.127.191:80",
"chrome.exe","4232","Console","1","79,432 K","Running","RMFVista\Admin","0:01:49","Network Security - Google Chrome"
"05.06.2010_21.30.33.60","74.125.127.105:443",
"chrome.exe","4232","Console","1","79,432 K","Running","RMFVista\Admin","0:01:49","Network Security - Google Chrome"
"05.06.2010_21.30.34.12","74.125.127.139:80",
"chrome.exe","4232","Console","1","79,432 K","Running","RMFVista\Admin","0:01:49","Network Security - Google Chrome"

ano_UDP.cmd output for UDP   
(note: No foreign IP addresses ever shows up in Microsoft's netstat for protocol UDP...as far as I can tell.)

"05.06.2010_21.29.42.51","*:*",
"nc.exe","4120","Console","1","572 K","Unknown","RMFVista\Admin","0:00:00","N/A"
"05.06.2010_21.29.44.07","*:*",
"svchost.exe","1196","Services","0","3,400 K","Unknown","NT AUTHORITY\LOCAL SERVICE","0:00:01","N/A"
"05.06.2010_21.29.44.50","*:*",
"svchost.exe","636","Services","0","52,188 K","Unknown","NT AUTHORITY\SYSTEM","0:07:36","N/A"
"05.06.2010_21.29.44.99","*:*",
"svchost.exe","636","Services","0","52,188 K","Unknown","NT AUTHORITY\SYSTEM","0:07:36","N/A"
"05.06.2010_21.29.45.42","*:*",
"svchost.exe","1288","Services","0","17,136 K","Unknown","NT AUTHORITY\NETWORK SERVICE","0:00:03","N/A"
"05.06.2010_21.29.45.87","*:*",
"VCSW.exe","5644","Services","0","3,540 K","Unknown","NT AUTHORITY\SYSTEM","0:00:04","N/A"

lsof (Linux 4.78) sample output
lsof -Ts | grep -i Firefox | grep IPv4
firefox 5756 root 5lu IPv4 22403 TCP 192.168.0.5:40814->nuq04s01-in-f113.le100.net:www (ESTABLISHED)

Day 2 at LinuxFest

Another great day at Linux Fest! I attended excellent presentations on Digital Forensics by Hal Pomeranz and Brian Pate (2 hours), both of which were very useful and felt very "hands on".  I can't say enough good things about LinuxFest. The organizers are doing Whatcom County business development a tremendous favor.  In reality, I think the Chamber of Commerce and the City of Bellingham should be helping to fund this volunteer supported event every quarter.  Talent comes from all over the Northwest: Seattle, Portland,  Tri-Cities, Olympia, Bothell, Mt. Vernon, you name it.  I made contacts, met vendors, passed out business cards and had great discussions. Learned a lot as well.

Thank you LinuxFest NorthWest!!!!

Brilliant Day 1 at LinuxFest NorthWest

I had a brilliant first day at LinuxFest NorthWest. I sat through five presentations on privacy and computer security in Haskell 115 at Bellingham Technical College. Brian Alseth of ACLU of Washington delivered the usual terrifying description of how data mining is  destroying privacy.  John Lock talked about Web Commerce Security. Gary Smith of PNL gave and excellent talk on Linux Server Hardening. Hal Pomeranz finished up the day with two hours on SE Linux. Wow! What a beast SE Linux is...

LinuxFest...a great thing.

Joanna Rutkowska and ITL and "Security by Isolation"

A day spent reading the research of Joanna Rutkowska and her Invisible Things Lab is a day spent improving your IQ. Ms. Rutkowska is famous for describing vulnerabilities in SMM, BIOS, and VM hypervisors.  In short, rather than attack the Operating System (although she has done some of that as well), she and her team attack the layer between the Operating System and the hardware; specifically rings -1, -2, -3 to use her terminology.  Her work has led her to some drastic conclusions about hardware and digital security.  In Joanna's universe, it is not that "game is over" but that the digital industry has never really fielded a team that could win yet. To do something about this, she and her team have developed a  customized version of Linux (Qubes-OS); partitioning off OS components into VMs to prevent the spread of malware through the access of "universal privilege" (my own term).

What do I mean by "universal privilege"? [Beware, the author's own untutored verbiage is to follow...] Computers are strange but beautiful machines.  When the first computational devices were built, we wanted to send in questions and retrieve answers. After computer scientists achieved this breakthrough, they spent the next half century attempting to generate increasing profits by increasing the speed at which answers to their questions would be returned. And they did a damn fine job at this. The increase in computational speed has to count as the single greatest technical advancement of our species by this point in history. Watch any movie about the Hubble or the Mars Rover and ask yourself: How would that happen without digital data? We have designed our computational efforts as if we were children with thirsty minds and ravenous social needs; ready to exercise our "universal privilege" to discuss/communicate/download whatever our minds and souls desire.

Security is mainly the story of protection. Secrecy is mainly the story of compartmentalization. In contrast to the development of computational speed, we've done a poor job at protection and compartmentalization of computers and their networks.  In fact, we've been so concerned about the spread of information, we've done everything possible to unleash the flow of digital data across the world.  PCs and Servers are now everywhere, in every complex product, in every country. Our computer networks are now the most tangible and real-time evidence of our civilization. Computers still retain all of the "strange and beautiful" architecture designed upon the premise that we want very little between our computers and fast answers to our questions.  We are by nature social creatures with unbounded curiosity and potentially unbounded need for "end to end" trust. Unfortunately, the reality of unconstrained digital response has helped created powerful  offensive weaponry in the untrustworthy world we live in.

So now back to universal privilege and Joanna Rutkowska and her team at Invisible Things Lab.  Eschewing (in part) the drive for secure code and secure micro-kernels, Joanna and her team attempt to do the following: 


"Qubes implements Security by Isolation approach. To do this, Qubes utilizes virtualization technology, to be able to isolate various programs from each other, and even sandbox many system-level components, like networking or storage subsystem, so that their compromise don’t affect the integrity of the rest of the system."


They achieve this "security by isolation" by compartmentalizing their OS into secure virtual machines. It is a  timely idea. As if to prove this, the NSF gave a $1.5 M dollar grant to an University of Illinois researcher nearly days after ITL's announcement of Qubes to do something similar. "Security by isolation" is an ancient concept thoroughly deployed by computer and software architecture at all levels. There are numerous examples: CPUs break down access to the processor into "Rings" (0-3).  Operating Systems break down execution in kernel and userland and then compartmentalize execution further. Some kernels just boot the most basic OS components, (Most desktop OS kernels are monolithic). Software compartmentalizes (perhaps 'componentizes') itself into functions, system calls, objects, and libraries. Some software, like Java and C#, works hard at making code live in a secure 'sandbox'. Part of the developmental reason for object oriented programming (originally) was (marginally) security-based: 'encapsulation'. Networking software has followed the trend of security by compartmentalization from switch fabric to firewalls to NAC. Hosted services, in some very real sense, are a form of "security by isolation".

In reality, we continue to invent "security by isolation" in kernels, software layers, networks, network components, firewalls, and virtual machines. As processor speed grows in an untrustworthy world, the desktop and network will always continue to need the most advanced compartmentalization to protect them from the expanding digitized world. To this end, our "universal privilege" to keep asking questions of each other will always be haunted by the necessity of  "security by isolation".

tcpslice II

More uses for tcpslice, ipsumdump, BASH 4.1 :

[This gives you today's top source IP and source IP Port combination: 


/usr/sbin/tcpslice `date +%Y"y"%m"m"%d"d"` $BASH_ARGV | ipsumdump --no-headers -sD - 

./todays_dump.sh MarApr.snort.in.tcpd | sort -nr | uniq -c | sort -nr
     13 85.144.201.237 7959
      3 95.179.99.147 5900
      3 64.206.157.2 23
      3 222.45.112.59 8085
      3 109.187.8.70 5900
      2 98.247.214.152 23 ...

This gives you today's top source IP and source IP location:


/usr/sbin/tcpslice `date +%Y"y"%m"m"%d"d"` $BASH_ARGV |
for i in `ipsumdump --no-headers -s -`
     do echo $i : $(printf "%s" `./geoip.sh $i | awk -F":" '{print $2}' | awk -F"," '{print $1","$2","$3}' ` ) 
done 

./tgeodump.sh MarApr.snort.in.tcpd | sort -nr | uniq -c | sort -nr
     13 85.144.201.237 : NL,07,Amsterdam
     12 222.45.112.59 : CN,22,Beijing
      4 222.215.230.49 : CN,32,Chengdu
      3 95.179.99.147 : RU,43,Lipetsk
      3 64.206.157.2 : US,NH,Nashua
      3 109.187.8.70 : IPAddressnotfound,,
      2 98.247.214.152 : US,WA,Bothell ...

where 'geoip.sh' is:
geoiplookup -f /usr/local/share/GeoIP/GeoLiteCity.dat $1


I note that file names like this '08Mar1142PST2010.in.1268074842' don't process through tcpslice.

tcpslice

Tcpslice is a useful tool from LBL network group that allows you to carve up a large pcap file format into time slices. 

To look at the start and finish time stamps of the entire pcap file in various time formats:

tcpslice -r Marchrferrisx.snort.in 

Marchrferrisx.snort.in  Mon Mar  8 11:08:09 2010        Mon Apr  5 09:09:37 2010

tcpslice -t Marchrferrisx.snort.in

Marchrferrisx.snort.in  2010y03m08d11h08m09s660222u     2010y04m05d09h09m37s390876u

tcpslice -R Marchrferrisx.snort.in

Marchrferrisx.snort.in  1268075289.660222       1270483777.390876

To return data from a particular time slice to a file with BPF filters use syntax like this: 

tcpslice 1257347146.060 1257347146.061 inputFile.tcpd | tcpdump -r - -w outputFile.tcpd 'host 192.168.1.175'

(Check out bothunter logs for more examples like this..)

In this example, I want all the packets that are not IPv6 for one date:

/usr/sbin/tcpslice 2010y04m05d Marchrferrisx.snort.in | /usr/sbin/tcpdump -r - 'not(ip6)' | less

reading from file -, link-type EN10MB (Ethernet)
01:06:17.290514 IP 125.141.195.190.35460 > 192.168.0.12.ssh: S 1607742099:1607742099(0) win 65535 
01:40:16.181816 IP c-98-247-214-152.hsd1.wa.comcast.net.catchpole > 192.168.0.12.telnet: SWE 498716114:498716114(0) win 5840
01:40:19.172942 IP c-98-247-214-152.hsd1.wa.comcast.net.catchpole > 192.168.0.12.telnet: SWE 498716114:498716114(0) win 5840
01:44:01.423708 IP hn.kd.ny.adsl.x11 > 192.168.0.12.ms-sql-s: S 833421312:833421312(0) win 16384
03:37:06.073237 IP 75.125.252.76.http > 192.168.0.12.48532: S 1175613974:1175613974(0) ack 143375003 win 14420
04:07:03.019711 IP 222.45.112.59.12200 > 192.168.0.12.ssm-els: S 363594672:363594672(0) win 8192 ...

Now I want all ms-sql-s destination packets from the ingress pcap that are not IPv6 for all of March:

/usr/sbin/tcpslice 2010y04m01d 2010y04m31d Marchrferrisx.snort.in | /usr/sbin/tcpdump -r - -n 'dst port(1433)'
reading from file -, link-type EN10MB (Ethernet)
18:33:42.614843 IP 125.46.78.100.x11 > 192.168.0.12.ms-sql-s: S 908984320:908984320(0) win 16384
23:38:50.771853 IP 61.183.172.35.x11 > 192.168.0.12.ms-sql-s: S 47316992:47316992(0) win 16384
03:35:18.351118 IP 121.12.125.7.x11 > 192.168.0.12.ms-sql-s: S 640548864:640548864(0) win 16384
11:09:45.631103 IP 218.61.127.71.x11 > 192.168.0.12.ms-sql-s: S 1613627392:1613627392(0) win 16384
00:47:21.207593 IP 218.90.163.66.x11 > 192.168.0.12.ms-sql-s: S 648937472:648937472(0) win 16384
08:56:05.732622 IP 61.183.172.35.x11 > 192.168.0.12.ms-sql-s: S 47316992:47316992(0) win 16384
18:06:53.798198 IP 59.51.114.39.x11 > 192.168.0.12.ms-sql-s: S 648937472:648937472(0) win 16384 ...
 

Something similar, but a little cleaner, can be done with ipsumdump:


/usr/sbin/tcpslice 2010y04m01d 2010y04m31d Marchrferrisx.snort.in | ipsumdump -tsD | grep -w 1433
 
1270172022.614843 125.46.78.100 1433 
1270190330.771853 61.183.172.35 1433 
1270204518.351118 121.12.125.7 1433 
1270231785.631103 218.61.127.71 1433 
1270280841.207593 218.90.163.66 1433 
1270310165.732622 61.183.172.35 1433 
1270343213.798198 59.51.114.39 1433 ...

One year anniversary

Today is the one year anniversary of this blog. This is my 48th post in that time period. According to Google Analytics, 1,250 “absolute unique visitors” have provided for 1,566 visits from 781 unique cities from 78 unique countries. 72 page titles were viewed a total of 2,241 times. Here are some of the most popular pages:

• /2009/07/parsing-vista-firewall-logs-part-iii.html
• /2009/05/i-receive-lots-of-6000-port-scans-on-my.html
• /2009/09/network-monitor-api-part-ii.html
• /2009/05/host-protection-working-with-microsofts.html
• /2009/09/ive-spent-last-three-weeks-building.html
• /2009/05/bash-40awkgeoiplookup-and-pcregrep-are.html
• /2009/05/brief-anatomy-of-malware-detection-and.html
• /2009/05/homegrown-tcpdumpsnort-analysis.html
• /2009/06/viisualizing-sips-over-timeportip-range.html

Actually, I have no idea what to make of any of these numbers.