PolyMorphic Multifunctional Spyware/Malware/Distribution Ware...

I have been writing scripts like those (far) below to parse my Firewall's syslogd output and wondering if there is some other, better way to understand the significance of my logs. Ultimately, I would like a little more information like: What causes the Firewall to behave as it does? Perhaps I am requesting a Firewall application log. For example, unidirectional SIPs are obviously blocked, but how does it know to continue bi-directional conversations across the WAN initiated from the LAN?  Sometimes I see an immediate block, other times I see a one packet delay.  Watching my firewall work drives home the point that the classic "firewall as perimeter defense" relinquishes control over what data escapes my network, especially if I do not employ host firewalls with bidirectional blocking capacity. 

And maybe even not then. The persistence of the current breed of spyware/malware/distribution ware (e.g. Conficker) is such that it can initiate web conversations on hidden, broken or validated channels (e.g register new domains) and then plunder host information from 'command and control' nodes once an internal node has established a validated external connection. Later the spyware/malware/distribution ware engages in 'polymorphic mutation' and updates itself to protect against updates in security products designed to detect it. The most notorious of such spyware/malware/distribution ware to date would be Conficker C. This is from http://mtc.sri.com/Conficker/addendumC/index.html:

"We present an analysis of Conficker Variant C, which emerged on the Internet at roughly 6 p.m. (PST) on 4 March 2009.  This variant incorporates significant new functionality, including a new domain generation algorithm and a new peer-to-peer file sharing service.   Absent from our discussion has been any reference to the well-known attack propagation vectors (RCP buffer overflow, USB, and NetBios Scans) that have allowed C's predecessors to saturate so much of the Internet.  Although not present in C, these attack propagation services are but one peer upload away from any C infected host, and may appear at any time. C is, in fact, a robust and secure distribution utility for distributing malicious content and binaries to millions of computers across the Internet.   This utility incorporates a potent arsenal of methods to defend itself from security products, updates, and diagnosis tools.  It further demonstrates the rapid development pace at which Conficker's authors are maintaining their current foothold on a large number of Internet-connected hosts.  Further, if organized into a coordinated offensive weapon, this multimillion-node botnet poses a serious and dire threat to the Internet."

I really don't think anyone has a clue of what to do about polymorphic, multifunctional spyware/malware/distribution-ware like this in terms of either prevention, detection, and perhaps removal. In fact, Conficker and other malware present a unique set of conundrums: Not just how do we detect this type of malware or how do we secure what we have already lost but how do we keep a multimillion node botnet from destroying the rest of the internet?  While the spyware/malware/distribution ware designers have propagated their creation throughout the universe, the rest of us are twiddling our fingers writing ditzy shell scripts in hopes they will tell us something about our Firewall's behaviour:

# more NotBlockedUniqLocation.sh                                                                                   

grep Dest message* | awk -F":" '{print $2$3$4 ":" "DIP:"$7":"}' | grep -v -f not_search.txt | grep -v [localhost] | awk -F":" '{print $3}' > NotBlocked.txt

for i in `cat NotBlocked.txt | sort -nr | uniq`; do echo $i `geoiplookup -f /usr/local/share/GeoIP/GeoLiteCity.dat $i | awk -F" " '{print $6 $7 $8 $9 }'`;done

# more BlockedUniqLocation.sh     

grep Blocked message* | awk -F":" '{print $2$3$4 ":" $6":"}' | grep -v -f not_search.txt | awk -F":" '{print $2}' > Blocked.txt

for i in `cat Blocked.txt | sort -nr | uniq`; do echo $i `geoiplookup -f /usr/local/share/GeoIP/GeoLiteCity.dat $i | awk -F" " '{print $6 $7 $8 $9}'`;done

# ./NotBlockedUniqLocation.sh 

222.172.105.109 CN,07,Zhongshan,(null),

217.109.17.57 FR,A8,Paris,(null),

216.73.87.115 US,NY,NewYork,

216.34.207.74 US,CA,WestlakeVillage,

216.34.181.91 US,CA,MountainView,

216.34.181.71 US,CA,MountainView,

216.34.181.60 US,CA,MountainView,

216.34.181.59 US,CA,MountainView,

216.34.181.40 US,CA,MountainView,

216.168.46.76 US,WA,Seattle,98168,

216.168.46.72 US,WA,Seattle,98168,

212.58.226.143 GB,P9,Maidenhead,(null),

212.58.226.142 GB,P9,Maidenhead,(null),

212.58.226.139 GB,P9,Maidenhead,(null),

209.66.102.82 US,CA,Berkeley,94709,

209.66.102.50 US,CA,Berkeley,94709,

...

# ./BlockedUniqLocation.sh                                                                                         

222.172.105.109 CN,07,Zhongshan,(null),

217.109.17.57 FR,A8,Paris,(null),

216.34.181.71 US,CA,MountainView,

216.34.181.60 US,CA,MountainView,

216.168.46.76 US,WA,Seattle,98168,

209.66.102.82 US,CA,Berkeley,94709,

209.66.102.50 US,CA,Berkeley,94709,

206.51.224.187 US,FL,Tampa,33602,

204.121.6.51 US,NM,LosAlamos,

203.146.189.74 TH,40,Bangkok,(null),

202.99.11.99 CN,22,Beijing,(null),

202.12.29.13 AU,04,Brisbane,(null),

199.93.55.126 US,(null),(null),(null),

199.43.0.144 US,(null),(null),(null),

199.212.0.43 CA,ON,Toronto,(null),

195.97.20.4 GR,35,Athens,(null),

192.149.252.44 US,(null),(null),(null),

168.75.68.60 US,MA,Andover,01810,

...