Bash 4.0,awk,geoiplookup and pcregrep are fast

Bash 4.0,awk,geoiplookup and pcregrep make a powerfully fast search team.  Here I find out how many sockets pairs are in my Snort dump: 

bash-4.0# snort -qdevX -r May12085154PDT2009.1242143514 | pcregrep TTL | awk -F":" '{print $1}' | wc -l                   

     372 

Then I find out how many uniqe SIPs are in those socket pairs:

bash-4.0# snort -qdevX -r May12085154PDT2009.1242143514 | pcregrep TTL | awk -F":" '{print $1}' | uniq | sort -nr | wc -l

     226 

Then I find my top ten Source IP Addresses:

bash-4.0# snort -qdevX -r May12085154PDT2009.1242143514 | pcregrep TTL | awk -F":" '{print $1}' | sort | uniq -c | sort -nr | head -n 10

  

  62 218.103.62.150

  15 222.215.230.49

  14 221.195.73.68

  11 121.15.245.215

  10 119.161.130.75

   8 209.85.163.126

   8 125.65.165.139

   7 66.35.46.195

   7 209.85.201.125

   6 64.106.128.150

....

Then I determine what source ports my top SIP (foreign address) has initiated connections to:

bash-4.0# snort -qdevX -r May12085154PDT2009.1242143514 | pcregrep 218.103.62.150 | awk -F":" '{print $3}'| awk -F" " '{print $1}' | sort -n | uniq

113

160

1433

1434

2967

5554

6429

7212

12712

16896

17337

17681

17919

18448

18487

18488

18649

18932

19899

33436

38507

Next I find what are the top source ports for my top ten SIPs: 

bash-4.0# for i in `cat temp.txt`; do echo $i && snort -qdevX -r May12085154PDT2009.1242143514 | pcregrep $i | awk -F":" '{print $3}'| awk -F" " '{print $1}' | sort | uniq -c | sort -nr; done

218.103.62.150

   3 6429

   3 5554

   3 38507

   3 33436

   3 2967

   3 19899

   3 18932

   3 18649

   3 18488

   3 18487

   3 18448

   3 17919

   3 17681

   3 17337

   3 16896

   3 160

   3 1434

   3 1433

   3 12712

   3 113

   2 7212

222.215.230.49

   6 7212

   6 3128

   3 8000

221.195.73.68

   7 8000

   7 7212

121.15.245.215

   7 3128

   4 8000

119.161.130.75

  10 2967

209.85.163.126

   8 33137

125.65.165.139

   4 8000

   4 3128

66.35.46.195

   7 33436

209.85.201.125

   3 18205

   3 18184

   1 18077

64.106.128.150

   6 33442

Last, I retrieve those top ten SIP city locations:

bash-4.0# for i in `cat temp.txt`; do echo $i `snort -qdevX -r May12085154PDT2009.1242143514 | geoiplookup $i -f /usr/local/share/GeoIP/GeoLiteCity.dat` ;done

218.103.62.150 GeoIP City Edition, Rev 1: HK, 00, Kowloon, (null), 22.316700, 114.183296, 0, 0

222.215.230.49 GeoIP City Edition, Rev 1: CN, 32, Chengdu, (null), 30.666700, 104.066597, 0, 0

221.195.73.68 GeoIP City Edition, Rev 1: CN, 10, Hebei, (null), 39.889702, 115.275002, 0, 0

121.15.245.215 GeoIP City Edition, Rev 1: CN, 30, Jiangmen, (null), 22.583300, 113.083298, 0, 0

119.161.130.75 GeoIP City Edition, Rev 1: CN, 19, Chaoyang, (null), 41.570301, 120.458603, 0, 0

209.85.163.126 GeoIP City Edition, Rev 1: US, CA, Mountain View, 94043, 37.419201, -122.057404, 807, 650

125.65.165.139 GeoIP City Edition, Rev 1: CN, 32, Chengdu, (null), 30.666700, 104.066597, 0, 0

66.35.46.195 GeoIP City Edition, Rev 1: US, CO, Denver, 80216, 39.785000, -104.941498, 751, 303

209.85.201.125 GeoIP City Edition, Rev 1: US, CA, Mountain View, 94043, 37.419201, -122.057404, 807, 650

64.106.128.150 GeoIP City Edition, Rev 1: US, NJ, Hoboken, 07030, 40.745800, -74.032097, 501, 201