cipherdyne.org

Michael Rash, Security Researcher



fwsnort    [Summary View]

Next »

Software Release - fwsnort-1.6.3

fwsnort-1.6.3 released The 1.6.3 release of fwsnort is available for download. This release adds a new test suite in the test/ directory that sends fwsnort through its paces for both iptables and ip6tables firewalls, speeds up iptables/ip6tables capabilities testing, and fixes a few bugs. In addition, one of the more significant changes is to ensure that Snort rules with HOME_NET=any -> EXTERNAL_NET=any are placed into the OUTPUT chain instead of the INPUT chain. This bug was reported by Dwight Davis. I would also like to thank Franck Joncourt for his support on the Debian side. Other changes were contributed by the open source community, and these are acknowledged in the complete fwsnort-1.6.3 ChangeLog below:

  • Bug fix to ensure that !, <, >, and = chars in content strings are converted to the appropriate hex equivalents. All content strings with characters outside of [A-Za-z0-9] are now converted to hex-string format in their entirety. This should also fix an issue that results in the following error when running /var/lib/fwsnort/fwsnort.sh:
        Using intrapositioned negation (`--option ! this`) is deprecated in
        favor of extrapositioned (`! --option this`).  Bad argument `bm'
        Error occurred at line: 64
        Try `iptables-restore -h' or 'iptables-restore --help' for more information.
        Done.
    
  • Bug fix to set default max string length in --no-ipt-test mode where iptables capabilities are not tested.
  • (Andrew Merenbach) Bug fix to properly honor --exclude-regex filtering option.
  • Added fwsnort test suite to the test/ directory. This mimics the test suites from the psad and fwknop projects, and it designed to examine many of the run time results of fwsnort.
  • Added the ability to easily revert the fwsnort policy back to the original iptables policy with "/var/lib/fwsnort/fwsnort.sh -r". Note that this reverts back to the policy as it was when fwsnort itself was executed.
  • Implemented a single unified function for iptables match parameter length testing, and optimized to drastically reduce run time for iptables capabilities checks (going from over 20 seconds to less than one second in some cases).
  • (Dwight Davis) Contributed patches for several bugs including not handling --exclude-regex properly, not ignoring the deleted.rules file, not handling --strict mode operations correctly, and more. These issues and the corresponding patch were originally reported here: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=693000
  • Bug fix for Snort rules with HOME_NET(any) -> EXTERNAL_NET(any) to ensure they go into the OUTPUT chain instead of the INPUT chain. This bug was reported by Dwight Davis.
  • Updated to bundle the latest Emerging Threats rule set.
The complete fwsnort-1.6.3 ChangeLog can also be found here via the fwsnort gitweb interface.

Software Release - fwsnort-1.6.2

fwsnort-1.6.2 released The 1.6.2 release of fwsnort is available for download. The most impactful change in this release is a switch to how fwsnort loads translated rules into the running iptables policy. Instead of attempting to parse the local policy and only add those rules in that appear to match protocols that the policy allows, fwsnort now loads all translated rules by default. The reasoning for this change is in the ChangeLog below. There are a few bug fixes and updates to get fwsnort working without warnings on recent versions of perl as well as an ICMP type fix for recent versions of iptables. As usual, please let me know if there are any issues.

Here is the complete fwsnort-1.6.2 ChangeLog:

  • Switched --no-ipt-sync to default to not syncing with the iptables policy. By default fwsnort attempts to match translated Snort rules to the running iptables policy, but this is tough to do well because iptables policies can be complex. And, before fwsnort switched to the iptables-save format for instantiating the policy, a large set of translated rules could take a really long time to make active within the kernel. Finally, many Snort rules restrict themselves to established TCP connections anyway, and if a restrictive policy doesn't allow connections to get into the established state for some port let's say, then there is little harm in having translated Snort rules for this port. Some kernel memory would be wasted (small), but no performance would be lost since packets won't be processed against these rules anyway. The end result is that the default behavior is now to not sync with the local iptables policy in favor of translating and instantiating as many rules as possible.
  • Replaced Net::IPv4Addr with the excellent NetAddr::IP module which has comprehensive support for IPv6 address network parsing and comparisons.
  • Moved the fwsnort.sh script and associated files into the /var/lib/fwsnort/ directory. This was suggested by Peter Vrabec.
  • Bug fix for recent versions of iptables (such as 1.4.12) where the icmp match requires --icmp-type to be set - some Snort rules look for a string to match in icmp traffic, but don't also specify an icmp type.
  • Bug fix for 'qw(...) usage as parenthesis' warnings for perl > 5.14
  • Removed the ExtUtils::MakeMaker RPM build requirement from the fwsnort.spec file. This is a compromise which will allow the fwsnort RPM to be built even if RPM doesn't or can't see that ExtUtils::MakeMaker is installed - most likely it will build anyway. If it doesn't, there are bigger problems since fwsnort is written in perl. If you want to build the fwsnort RPM with a .spec file that requires ExtUtils::MakeMaker, then use the "fwsnort-require-makemaker.spec" file that is bundled in the fwsnort sources.

Software Release - fwsnort-1.6

fwsnort-1.6 released The 1.6 release of fwsnort is available for download. This is a fairly significant release that adds support for the Snort fast_pattern keyword, makes enhancements to the --QUEUE and --NFQUEUE modes, supports the conntrack module for connection tracking, adds support for case-insensitive pattern matches using the --icase argument to the iptables string match extension, and several other things. The Snort fast_pattern keyword allows the rule author to influence the order in which Snort tries to match a pattern against network traffic. When multiple patterns are included in a rule, Snort usually tries to match the longest pattern first reasoning that the longest pattern is probably the least likely to trigger a match and therefore the remaining pattern searches would not have to be performed. But, there are times when the rule author would like to explicitly ask Snort to match on a particular pattern first, and the fast_pattern keyword is the mechanism that makes this possible. Because iptables matches are evaluated in order and a failing match short circuits a rule, fast_pattern support with the string match extension is possible through proper ordering in the iptables rule. Here is an example Snort rule from Emerging Threats with the fast_pattern keyword applied to the forth pattern: alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"ET WEB_CLIENT Possible Internet Explorer srcElement Memory Corruption Attempt"; flow:established,to_client; file_data; content:"document.createEventObject"; distance:0; nocase; content:".innerHTML"; within:100; nocase; content:"window.setInterval"; distance:0; nocase; content:"srcElement"; fast_pattern; nocase; distance:0; classtype:attempted-user; reference:url,www.microsoft.com/technet/security/bulletin/ms10-002.mspx; reference:url,tools.cisco.com/security/center/viewAlert.x?alertId=19726; reference:url,www.kb.cert.org/vuls/id/492515; reference:cve,2010-0249; reference:url,doc.emergingthreats.net/2010799; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/CURRENT_EVENTS/CURRENT_MSIE; sid:2010799; rev:5;) fwsnort translates this rule as follows in iptables-save format from the /etc/fwsnort/fwsnort.save file - the original iptables commands in non-save format are also available in the /etc/fwsnort/fwsnort_iptcmd.sh script: -A FWSNORT_FORWARD_ESTAB -p tcp -m tcp --sport 80 -m string --string "srcElement" --algo bm --from 82 --icase -m string --string "document.createEventObject" --algo bm --from 64 --icase -m string --string ".innerHTML" --algo bm --to 190 --icase -m string --string "window.setInterval" --algo bm --from 74 --icase -m comment --comment "sid:2010799; msg:ET WEB_CLIENT Possible Internet Explorer srcElement Memory Corruption Attempt; classtype:attempted-user; reference:url,www.microsoft.com/technet/security/bulletin/ms10-002.mspx; rev:5; FWS:1.6;" -j LOG --log-ip-options --log-tcp-options --log-prefix "SID2010799 ESTAB " Note that the srcElement string is matched first in the iptables rule even though it is the last string in the original Snort rule. With fast_pattern support, fwsnort policies should be a bit faster at run time in the in the Linux kernel. On a final note, the iptables multiport match is also supported with the fwsnort-1.6 release, so Snort rules with lists of source or destination ports (like this: "alert tcp $HOME_NET [0:20,22:24,26:138,140:444,446:464,466:586,588:901,903:1432,1434:65535] -> any any") can be translated.

The complete fwsnort-1.6 ChangeLog can be found here via the fwsnort gitweb interface.

Software Release - fwsnort-1.5

fwsnort-1.5 released The 1.5 release of fwsnort is available for download. This is a major release that moves to using the iptables-save format for instantiating the fwsnort policy, and this allows the run time for adding the fwsnort policy to the kernel to be drastically reduced. fwsnort now splices in the translated Snort rules into the iptables policy in the running kernel at the time of translation. So, any updates to the iptables policy that are made after fwsnort is executed and before fwsnort.sh is run would be lost. Hence, it is advisable to execute fwsnort.sh soon after running fwsnort. This is a reasonable tradeoff though considering the performance benefit as seen below - which gives an example of how long it takes to add an fwsnort iptables policy via the old strategy of executing one iptables command at a time vs. implementing the same policy with iptables-restore. First, fwsnort is used to translate the Snort web-misc.rules, web-cgi.rules, backdoor.rules files like so: [root@minastirith /etc/fwsnort]# fwsnort --snort-rfile web-misc.rules,web-cgi.rules,backdoor.rules --no-ipt-sync

[+] Generated iptables rules for 713 out of 754 signatures: 94.56%

[+] Logfile: /var/log/fwsnort/fwsnort.log
[+] iptables script (individual commands): /etc/fwsnort/fwsnort_iptcmds.sh

Main fwsnort iptables-save file: /etc/fwsnort/fwsnort.save

You can instantiate the fwsnort policy with the following command:

/sbin/iptables-restore < /etc/fwsnort/fwsnort.save

Or just execute: /etc/fwsnort/fwsnort.sh
The output above illustrates the changes for the fwsnort-1.5 release. All of the previous behavior in fwsnort has been preserved in the /etc/fwsnort/fwsnort_iptcmds.sh script. That is, each individual iptables command to add every fwsnort rule one by one is implemented in this script - this is analogous to how the fwsnort.sh script was built by older versions of fwsnort. But, with the 1.5 release the fwsnort.sh script now just executes the iptables-restore command against the new fwsnort.save file.

If we execute the fwsnort_iptcmds.sh script and time its execution, we get the following on my desktop system: [root@minastirith /etc/fwsnort]# time /etc/fwsnort/fwsnort_iptcmds.sh
[+] Adding backdoor rules:
Rules added: 122
[+] Adding web-cgi rules:
Rules added: 696
[+] Adding web-misc rules:
Rules added: 600
[+] Finished.

real 0m24.391s
user 0m1.560s
sys 0m11.500s
[root@minastirith /etc/fwsnort]# iptables -v -nL |wc -l
1509
So, the fwsnort policy together with the running iptables policy is about 1500 rules long, and it took over 24 seconds to add to the running kernel. Now, let's time the fwsnort.sh script instead (which is just a wrapper around the iptables-restore command): [root@minastirith /etc/fwsnort]# time /etc/fwsnort/fwsnort.sh

[+] Splicing fwsnort rules into the iptables policy...
Done.


real 0m0.121s
user 0m0.060s
sys 0m0.040s
[root@minastirith /etc/fwsnort]# iptables -v -nL |wc -l
1509
Ok, over 24 seconds to instantiate the fwsnort policy for the old strategy, and about a 10th of a second for the new strategy for a speed up of 240 times! This gets even better for an fwsnort policy with thousands of rules. Note that the number of iptables rules is the same between the two executions.

The complete ChangeLog entries are displayed below:

  • Major update to use the iptables-save format instead of the older strategy of always just executing iptables commands directly (which was very flow for large fwsnort policies). The /etc/fwsnort/fwsnort.sh script now just executes:

    /sbin/iptables-restore < /etc/fwsnort/fwsnort.save

    All fwsnort rules are now placed in the /etc/fwsnort/fwsnort.save file, but the older fwsnort.sh output (for the individual commands version) is still available at /etc/fwsnort/fwsnort_iptcmds.sh. This functionality extends to ip6tables policies as well. The fwsnort man page explain this in better detail:

    "As of fwsnort-1.5 all iptables rules built by fwsnort are written out to the /etc/fwsnort/fwsnort.save file in iptables-save format. This allows a long fwsnort policy (which may contain thousands of iptables rules translated from a large Snort signature set) to be quickly instantiated via the "iptables-restore" command. A wrapper script /etc/fwsnort/fwsnort.sh is also written out to make this easy. Hence, the typical work flow for fwsnort is to: 1) run fwsnort, 2) note the Snort rules that fwsnort was able to successfully translate (the number of such rules is printed to stdout), and then 3) execute the /etc/fwsnort/fwsnort.sh wrapper script to instantiate the policy in the running kernel."
  • Added the --rules-url argument so that the URL for updating the Emerging Threats rule set can be specified from the command line. The default is:

    http://rules.emergingthreats.net/open/snort-2.9.0/emerging-all.rules
  • Updated to automatically check for the maximum length string that the string match supports, and this is used to through out any Snort rules with content matches longer than this length.
  • Updated the iptables capabilities testing routines to add and delete testing rules to/from the custom chain 'FWS_CAP_TEST'. This maintains a a cleaner separation between fwsnort and any existing iptables policy even during the capabilities testing phase.
  • Added the --ipt-check-capabilities argument to have fwsnort test the capabilities of the local iptables firewall and exit.
  • Added the --string-match-alg argument to allow the string matching algorithm used by fwsnort to be specified from the command line. The default algorithm is 'bm' for 'Boyer-Moore', but 'kmp' may also be specified (short for the 'Knuth-Morris-Pratt' algorithm).
  • Updated to the latest complete rule set from Emerging Threats (see http://www.emergingthreats.net/).

How to avoid ClamAV matches on bundled Snort rules

ClamAV matches on Snort rules Recently a psad user notified me via email that the psad-2.1.7.tar.gz tarball was flagged by ClamAV as being infected with Exploit.HTML.MHTRedir-8. After checking a few things, it turns out that ClamAV is triggering on a Snort rule in the Emerging Threats rule set which is bundled in both psad and fwsnort. The following analysis shows exactly what ClamAV is detecting and why, and also provides some guidance for how to avoid this for any software projects that distribute Snort rules. Similar logic would apply to other software engineering efforts - including commercial intrusion detection systems - that are (by their nature) looking for malicious artifacts on the filesystem or within network traffic.

First, let's download psad-2.1.7.tar.gz and check the gpg signature (just to make sure we're talking about exactly the same file): $ wget --quiet http://www.cipherdyne.org/psad/download/psad-2.1.7.tar.gz
$ wget --quiet http://www.cipherdyne.org/psad/download/psad-2.1.7.tar.gz.asc
$ gpg --verify psad-2.1.7.tar.gz.asc
gpg: Signature made Wed 14 Jul 2010 06:01:42 PM EDT using DSA key ID 0D3E7410
gpg: Good signature from "Michael Rash (Signing key for cipherdyne.org projects) <mbr@cipherdyne.org>"
Ok, now here is what clamscan says about the psad-2.1.7.tar.gz tarball: $ clamscan psad-2.1.7.tar.gz
psad-2.1.7.tar.gz: Exploit.HTML.MHTRedir-8 FOUND

----------- SCAN SUMMARY -----------
Known viruses: 816934
Engine version: 0.96.1
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 6.42 MB
Data read: 1.16 MB (ratio 5.55:1)
Time: 7.169 sec (0 m 7 s)
Let's see which file within the psad-2.1.7 tarball matches the Exploit.HTML.MHTRedir-8 signature: $ tar xfz psad-2.1.7.tar.gz
$ clamscan -r -i psad-2.1.7
psad-2.1.7/deps/snort_rules/emerging-all.rules: Exploit.HTML.MHTRedir-8 FOUND

----------- SCAN SUMMARY -----------
Known viruses: 816934
Engine version: 0.96.1
Scanned directories: 41
Scanned files: 405
Infected files: 1
Data scanned: 12.55 MB
Data read: 6.41 MB (ratio 1.96:1)
Time: 8.446 sec (0 m 8 s)
Intuitively, this makes sense. That is, given that ClamAV is out to identify nasty things within files, and given that Snort rules are designed to identify nasty things as they communicate over the network, it stands to reason that there might be some overlap. This overlap is not an indication of something wrong in either the Snort rules or in ClamAV. Now, let's find out specifically which Snort rule within the emerging-all.rules file is triggering the ClamAV match. We first take a look at the Exploit.HTML.MHTRedir-8 signature: $ cp /var/lib/clamav/main.cvd .
$ sigtool --unpack main.cvd
$ grep Exploit.HTML.MHTRedir-8 main.ndb
Exploit.HTML.MHTRedir-8:3:*:6d68746d6c3a66696c653a2f2f{1-20}2168
The last line above is the entire ClamAV signature, and the pattern 6d68746d6c3a66696c653a2f2f is the key. The ":3:" part identifies the signature as type "normalized HTML", so ClamAV matches the pattern 6d68746d6c3a66696c653a2f2f against the "normalized HTML" representation of each processed file. We can decode the pattern as follows: echo 6d68746d6c3a66696c653a2f2f | xxd -r -p
mhtml:file://
So, within the emerging-all.rules file, we are interested in any Snort rule that contains the string mhtml:file://. There is also the "{1-20}2168" criteria which says to match the hex bytes 2168 anywhere from 1 to 20 bytes after the first pattern match. $ grep mhtml psad-2.1.7/deps/snort_rules/emerging-all.rules
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET MALWARE Bundleware Spyware CHM Download"; flow: to_server,established; content:"Referer\: ms-its\:mhtml\:file\://C\:counter.mht!http\://"; nocase; content:"/counter/HELP3.CHM\:\:/help.htm"; nocase; classtype: trojan-activity; sid: 2001452; rev:4;)
Sure enough, sid:2001452 "ET MALWARE Bundleware Spyware CHM Download" has the keyword content:"Referer\: ms-its\:mhtml\:file\://C\:counter.mht!http\://". Even though there are escaping backslashes, the normalized HTML processing in ClamAV takes this into account and matches the pattern anyway from the ClamAV signature.

So, how can we keep the original Snort rule, but change it so that ClamAV not longer flags it?

Fortunately, ClamAV does not interpret the Snort rules convention of specifying non-printable bytes between "|" characters within content fields, so we simply need to change one of characters to hex notation. Snort will still offer the same detection if network traffic matches the rule, and ClamAV won't flag it. So, let's just change the "m" in "mhtml\:file\://" to its hex equivalent, like so: "|6d|html\:file\://". Once we make this change and save the psad-2.1.7/deps/snort_rules/emerging-all.rules file, we rerun clamscan: $ clamscan -r -i psad-2.1.7

----------- SCAN SUMMARY -----------
Known viruses: 816934
Engine version: 0.96.1
Scanned directories: 41
Scanned files: 405
Infected files: 0
Data scanned: 17.23 MB
Data read: 6.41 MB (ratio 2.69:1)
Time: 10.024 sec (0 m 10 s)
That's better. Over the next few days I'll re-release all affected versions of psad and fwsnort with the above change to ensure that there aren't any additional ClamAV matches.

In conclusion, if you are involved in any software engineering effort that distributes or makes use of Snort rules, it is probably a good idea to run distribution packages through ClamAV and see if there are any matches. If so, it may be possible to take advantage of Snort rule syntax options to still achieve the same signature coverage while not having ClamAV flag anything.

fwsnort Available in Fedora Repositories

fwsnort on Fedora The fwsnort project is now available directly through the Fedora RPM repositories (for Fedora 12 and 13) thanks to Guillermo Gomez. The version that is currently bundled is fwsnort-1.0.6. Once fwsnort-1.2 is released, the implementation of large iptables rule sets that are derived from Snort rules will become a lot faster. This is because fwsnort is going to support the iptables-save format by integrating the complex rules built by fwsnort with any existing iptables policy that is instantiated in the kernel. This is made possible by interpreting the local policy and splicing in all of the fwsnort rules in the right places - each iptables chain is built from scratch upon an iptables-restore (including the built-in chains), so integrating with a running policy is not as easy as just adding each fwsnort rule into a set of custom chains. Compatibility with the iptables-save format has largely been completed with this patch in the fwsnort-1.2 development effort.

If you are running a Fedora 12 or 13 system, you can install fwsnort like so: # yum install fwsnort

Software Release - fwsnort-1.1

software release fwsnort-1.1 The 1.1 release of fwsnort is ready for download. This is a significant release that adds support for ip6tables so that SNORT ® inspection logic can be applied to IPv6 traffic within the Linux kernel. This release also includes a new feature that allows fwsnort to build perl commands interfaced with netcat that generate packet data that matches Snort rules (for those that that can be faithfully translated - see the --include-perl-triggers command line argument and associated comments within the fwsnort.sh file).

Here is the complete fwsnort-1.1 ChangeLog:

  • Added the ability to build an fwsnort policy that utilizes ip6tables instead of iptables. This allows fwsnort filtering and altering capabilities to apply to IPv6 traffic instead of just IPv4 traffic. To enable ip6tables usage, use the "-6" or "--ip6tables" command line arguments.
  • Added the --include-perl-triggers command line argument so that translated Snort rules can easily be tested. This argument instructs fwsnort to include 'perl -e print ... ' commands as comments in the /etc/fwsnort/fwsnort.sh script, and these commands can be combined with netcat to send payloads across the wire that match Snort rules.
  • Updated fwsnort to create logs in the /var/log/fwsnort/ directory instead of directly in the /var/log/ directory. The path is controlled by a new variable 'LOG_FILE' in the /etc/fwsnort/fwsnort.conf file.
  • Added several variables in /etc/fwsnort/fwsnort.conf to control paths to everything from the config file to the snort rules path. Coupled with this is the ability to create variables within path components and fwsnort will expand them (e.g. 'CONF_DIR /etc/fwsnort; CONF_FILE $CONF_DIR/fwsnort.conf').
  • Added --Last-cmd arg so that it is easy to rebuild the fwsnort.sh script with the same command line args as the previous execution.
Snort is a registered trademark of Sourcefire, Inc.

Disrupting Conficker Worm Traffic with iptables and fwsnort

fwsnort vs. Conficker Although the media blitz surrounding the Conficker worm has died down, the worldwide computing infrastructure that the worm has cobbled together still exists and remains under the control of its masters. The resulting botnet is an impressive demonstration of distributed computing control and recoverability. Many organizations - from companies to governments - would be envious of such automation. Most likely the botnet is being used as a money making machine by renting out "botnet time" to criminals who then use it for their own purposes. New Scientist has a good summary of the Conficker saga, and includes a discussion of its switch from HTTP to a peer-to-peer module for communications and updates. Even though Conficker has perhaps not yet been used to issue DoS attacks against high profile sites, it has had measurable impacts such as leaving Manchester unable to issue parking tickets and Microsoft announcing a $250,000 bounty on the Conficker authors. On the defense side, the Conficker Working Group has produced some nice infection distribution maps and Nmap added Conficker scan detection based on an excellent paper written by Tillmann Werner and Felix Leder.

In the context of iptables and fwsnort, the goal is to give Linux systems the ability to detect and interfere network traffic associated with Conficker (at least as much as possible), and this process starts with Snort rules from the Emerging Threats rule set. There are currently six active Snort rules designed to detect Conficker in the Emerging Threats set, and an additional four that have been commented out. The six active rules so far detect the Conficker.A and Conficker.B variants, but hopefully more rules will become available as better detection techniques are developed.

Now, how does fwsnort do with translating the six Emerging Threats rules? Let's find out with the command below. This uses the --include-regex feature to restrict fwsnort to just those rules that contain the string "conficker", and we also add the new --include-perl-trigger argument (to be released in fwsnort-1.0.7) that builds a perl command to mimic the application layer data in each Snort rule. By combining this perl command with netcat, it is possible to test whether the iptables policy built by fwsnort properly detects attacks. Finally, we also use the --ipt-reject argument to have iptables drop any packet that matches the Conficker signatures and reset the connection at the same time: # fwsnort --include-regex conficker --include-re-caseless --snort-rfile /etc/fwsnort/snort_rules/emerging-all.rules --include-perl-triggers --ipt-reject | tail -n 4
[+] Generated iptables rules for 3 out of 6 signatures: 50.00%

[+] Logfile: /var/log/fwsnort.log
[+] iptables script: /etc/fwsnort/fwsnort.sh
Ok, so three out of the six signatures (I'm using 'signature' and 'rule' interchangeably in this blog post) converted properly to iptables rules. Those that did not convert contain elements such as pcre and threshold that are not currently supported by fwsnort.

Below is an example of one Snort signature that did convert correctly. This is rule ID 2009201, and it detects shellcode directed at TCP/445 from Conficker.B: alert tcp $EXTERNAL_NET any -> $HOME_NET 445 (msg:"ET CURRENT_EVENTS Conficker.b Shellcode"; flow:established,to_server; content:"|e8 ff ff ff ff c2|_|8d|O|10 80|1|c4|Af|81|9MSu|f5|8|ae c6 9d a0|O|85 ea|O|84 c8|O|84 d8|O|c4|O|9c cc|Ise|c4 c4 c4|,|ed c4 c4 c4 94|&<O8|92|\;|d3|WG|02 c3|,|dc c4 c4 c4 f7 16 96 96|O|08 a2 03 c5 bc ea 95|\;|b3 c0 96 96 95 92 96|\;|f3|\;|24 |i|95 92|QO|8f f8|O|88 cf bc c7 0f f7|2I|d0|w|c7 95 e4|O|d6 c7 17 cb c4 04 cb|{|04 05 04 c3 f6 c6 86|D|fe c4 b1|1|ff 01 b0 c2 82 ff b5 dc b6 1f|O|95 e0 c7 17 cb|s|d0 b6|O|85 d8 c7 07|O|c0|T|c7 07 9a 9d 07 a4|fN|b2 e2|Dh|0c b1 b6 a8 a9 ab aa c4|]|e7 99 1d ac b0 b0 b4 fe eb eb|"; reference:url,www.honeynet.org/node/388; reference:url,doc.emergingthreats.net/2009201; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/CURRENT_EVENTS/CURRENT_Conficker; classtype:trojan-activity; sid:2009201; rev:4;) Here is the equivlent iptables command built by fwsnort and included in the /etc/fwsnort/fwsnort.sh script. Note the usage of the FWSNORT_FORWARD_ESTAB chain which is reserved for packets that are part of established TCP connections: $IPTABLES -A FWSNORT_FORWARD_ESTAB -p tcp --dport 445 -m string --hex-string "|e8ffffffffc2|_|8d|O|1080|1|c4|Af|81|9MSu|f5|8|aec69da0|O|85ea|O|84c8|O|84d8|O|c4|O|9ccc|Ise|c4c4c4|,|edc4c4c494|&<O8|923bd3|WG|02c3|,|dcc4c4c4f7169696|O|08a203c5bcea953bb3c096969592963bf33b24|i|9592|QO|8ff8|O|88cfbcc70ff7|2I|d0|w|c795e4|O|d6c717cbc404cb|{|040504c3f6c686|D|fec4b1|1|ff01b0c282ffb5dcb61f|O|95e0c717cb|s|d0b6|O|85d8c707|O|c0|T|c7079a9d07a4|fN|b2e2|Dh|0cb1b6a8a9abaac4|]|e7991dacb0b0b4feebeb|" --algo bm -m comment --comment "sid:2009201; msg:ET CURRENT_EVENTS Conficker.b Shellcode; classtype:trojan-activity; reference:url,www.honeynet.org/node/388; rev:4; FWS:1.0.6;" -j LOG --log-ip-options --log-tcp-options --log-prefix "[3] DRP SID2009201 ESTAB " Because the pattern in the above signature is longer than 128 bytes, we'll increase the value of the MAX_STRING_LEN variable to 256 in the /etc/fwsnort/fwsnort.conf file. With that done, let's execute the /etc/fwsnort/fwsnort.sh script now and see how iptables handles such traffic on the wire: # /etc/fwsnort/fwsnort.sh
[+] Adding emerging-all rules:
iptables v1.4.1.1: STRING too long `|e8ffffffffc2|_|8d|O|1080|1|c4|Af|81|9MSu|f5|8|aec69da0|O|85ea|O|84c8|O|84d8|O|c4|O|9ccc|Ise|c4c4c4|,|edc4c4c494|&<O8|923bd3|WG|02c3|,|dcc4c4c4f7169696|O|08a203c5bcea953bb3c096969592963bf33b24|i|9592|QO|8ff8|O|88cfbcc70ff7|2I|d0|w|c795e4|O|d6c717cbc404cb|{|040504c3f6c686|D|fec4b1|1|ff01b0c282ffb5dcb61f|O|95e0c717cb|s|d0b6|O|85d8c707|O|c0|T|c7079a9d07a4|fN|b2e2|Dh|0cb1b6a8a9abaac4|]|e7991dacb0b0b4feebeb|' Try `iptables -h' or 'iptables --help' for more information.
Ok, that is disappointing. It turns out that iptables currently enforces a 128-byte maximum on all strings supplied to the string match extension for inspecting payload data. Normally this is not a problem since the individual patterns in most Snort rules are typically less than 128 bytes, but in this case we'd like to work around this limitation. To do so requires that we patch and recompile the xt_string kernel module (assuming xt_string is configured as a module) with the following patch: # git diff
diff --git a/include/linux/netfilter/xt_string.h b/include/linux/netfilter/xt_string.h
index 8a6ba7b..afc60a2 100644
--- a/include/linux/netfilter/xt_string.h
+++ b/include/linux/netfilter/xt_string.h
@@ -1,7 +1,7 @@
#ifndef _XT_STRING_H
#define _XT_STRING_H

-#define XT_STRING_MAX_PATTERN_SIZE 128
+#define XT_STRING_MAX_PATTERN_SIZE 256
#define XT_STRING_MAX_ALGO_NAME_SIZE 16

enum {
@@ -15,7 +15,7 @@ struct xt_string_info
u_int16_t to_offset;
char algo[XT_STRING_MAX_ALGO_NAME_SIZE];
char pattern[XT_STRING_MAX_PATTERN_SIZE];
- u_int8_t patlen;
+ u_int16_t patlen;
union {
struct {
u_int8_t invert;
With the new xt_string module loaded let's execute the fwsnort.sh script once again: # /etc/fwsnort/fwsnort.sh
[+] Adding emerging-all rules:
Rules added: 12
[+] Finished.
Ah, that's better. The fwsnort iptables policy loaded properly in the running kernel. Now, let's use the perl trigger command along with netcat to send data across the wire that should match the signature. The trigger itself can be found in the /etc/fwsnort/fwsnort.sh script. First, we fire up a netcat server on TCP port 445 on a target system which is protected by another system running the fwsnort iptables policy, and then with the perl trigger we send bytes that match the Conficker.B shell code signature across the wire to the target. The complete perl command is listed below even though it certainly is obtuse looking. You can see how the bytes it is printing match the content strings in the original signature: [target]# nc -l -p 445
[attacker]$ perl -e 'print "\xe8\xff\xff\xff\xff\xc2_\x8dO\x10\x801\xc4Af\x819MSu\xf58\xae\xc6\x9d\xa0O\x85\xeaO\x84\xc8O\x84\xd8O\xc4O\x9c\xccIse\xc4\xc4\xc4,\xed\xc4\xc4\xc4\x94&<O8\x92\x3b\xd3WG\x02\xc3,\xdc\xc4\xc4\xc4\xf7\x16\x96\x96O\x08\xa2\x03\xc5\xbc\xea\x95\x3b\xb3\xc0\x96\x96\x95\x92\x96\x3b\xf3\x3b\x24i\x95\x92QO\x8f\xf8O\x88\xcf\xbc\xc7\x0f\xf72I\xd0w\xc7\x95\xe4O\xd6\xc7\x17\xcb\xc4\x04\xcb{\x04\x05\x04\xc3\xf6\xc6\x86D\xfe\xc4\xb11\xff\x01\xb0\xc2\x82\xff\xb5\xdc\xb6\x1fO\x95\xe0\xc7\x17\xcbs\xd0\xb6O\x85\xd8\xc7\x07O\xc0T\xc7\x07\x9a\x9d\x07\xa4fN\xb2\xe2Dh\x0c\xb1\xb6\xa8\xa9\xab\xaa\xc4]\xe7\x99\x1d\xac\xb0\xb0\xb4\xfe\xeb\xeb"' |nc 10.1.1.1 445
The fwsnort iptables policy has reset the connection, and the following iptables log message was also produced: Jul 4 13:23:18 fwsnort kernel: [10966.350782] [2] REJ SID2009201 ESTAB IN=lo OUT= MAC=AB:00:00:AB:00:00:AB:00:00:AB:00:00:08:00 SRC=192.168.10.1 DST=10.1.1.1 LEN=244 TOS=0x00 PREC=0x00 TTL=64 ID=5976 DF PROTO=TCP SPT=49053 DPT=445 WINDOW=513 RES=0x00 ACK PSH URGP=0 OPT (0101080A0028B05B0028B058) Of course, the best defense against Conficker is to patch Windows systems against the MS08-067 vulnerability, and to use Nmap to scan for systems that are already infected. Those that are should be completely reimaged.

Software Release - fwsnort-1.0.6

software release fwsnort-1.0.6 The 1.0.6 release of fwsnort is ready for download. This release fixes a bug that caused some Snort rules to not be translated into iptables rules due to improper handling of escaped semicolons. Now that this bug has been fixed, an additional 58 rules from the Emerging Threats rule set are now properly supported. Also made it easier to point fwsnort at a single file with a Snort rule set to be converted (see the --fwsnort-rfile command line argument).

Here is the complete ChangeLog:

  • (Franck Joncourt) Updated fwsnort to use the "! <option> <arg> syntax instead of the older "<option> ! <arg> for the iptables command line.
  • (Franck Joncourt) For the --hex-string and --string matches, if the argument exceeds 128 bytes (iptables 1.4.2) then iptables fails with an error "iptables v1.4.2: STRING too long". Fixes this with a patch that adds a new variable in fwsnort.conf "MAX_STRING_LEN", so that the size of the content can be limited. If the content (null terminated string) is more than MAX_STRING_LEN chars, fwsnort throws the rule away.
  • Bug fix to allow fwsnort to properly translate snort rules that have "content" fields with embedded escaped semicolons (e.g. "\;"). This allows fwsnort to translate about 58 additional rules from the Emerging Threats rule set.
  • Bug fix to allow case insensitive matches to work properly with the --include-re-caseless and --exclude-re-caseless arguments.
  • Bug fix to move the 'rawbytes' keyword to the list of keywords that are ignored since iptables does a raw match anyway as it doesn't run any preprocessors in the Snort sense.
  • Added the --snort-rfile argument so that a specific Snort rules file (or list of files separated by commas) is parsed.
  • Added a small hack to choose the first port from a port list until the iptables 'multiport' match is supported.
  • Updated to consolidate spaces in hex matches in the fwsnort.sh script since the spaces are not part of patterns to be searched anyway.
  • Updated to the latest complete rule set from Emerging Threats (see http://www.emergingthreats.net/).
  • Added the "fwsnort-nobuildreqs.spec" file for building fwsnort on systems (such as Debian) that do not install/upgrade software via RPM. This file omits the "BuildRequires: perl-ExtUtils-MakeMaker" directive, and this fixes errors like the following on an Ubuntu system when building fwsnort with rpmbuild: rpm: To install rpm packages on Debian systems, use alien. See README.Debian.
    error: cannot open Packages index using db3 - No such file or directory (2)
    error: cannot open Packages database in /var/lib/rpm

Handling Escaped Semicolons in Snort Rules with fwsnort

fwsnort and escaped semicolons in Snort rules Recently I ran into a situation in which several Snort rules from the Emerging Threats rule sets were not being properly translated into iptables rules by fwsnort. It turned out that fwsnort did not correctly parse Snort content fields that contained escaped semicolons (e.g. "\;"). In the Snort signature language, the argument to every keyword in the body of a Snort rule such as content, pcre, and flowbits is terminated with a semicolon, and some keywords also use opening and closing double quotes. But, Snort supports escaping with a backslash so that these characters can easily be made to be part of a keyword argument as opposed to the delimiting syntax. Snort does not allow the argument of a content keyword to contain an embedded semicolon that is not escaped (e.g. content:"distloc=;";), and will generate an error similar to the following if a rule does not conform to this: Initializing rule chains...
ERROR: /etc/snort/rules/web-cgi.rules(3) =& Content data needs to be enclosed in quotation marks (")!
Fatal Error, Quitting..
In this case, we change content:"distloc=;"; to content:"distloc=\;"; and the error goes away. However, in addition to the escaping mechanism, any double quote or semicolon that is part of a content field can just be specified in hex notation between pipe "|" characters instead.

So, what are the tradeoffs in using one convention vs. the other?

Using backslashes can complicate the way an argument looks (since backslashes are not part of the content that is actually searched for in network traffic), but they can also make the argument more intuitive to look at than the hex syntax. This can be important when looking at lots of packet traces. For example, in web traffic the semicolon is used in HTTP request headers as a separator and therefore has special significance in HTTP, and the semicolon is also a separator for multiple commands launched from a command shell. So, for those that don't automatically know the hex equivalent of a semicolon (0x3b), it might be better to look at content:"distloc=\;"; instead of content:"distloc=|3B|"; when interpreting signature matches against raw packet traces since it emphasizes the importance of the semicolon.

There are important examples of Snort rule sets that use each strategy for the arguments to content fields (escaped semicolons vs. the hex equivalent). The complete Emerging Threats rule set contains 58 signatures with escaped semicolons: $ perl -lwne 'while (/content:"(.*?)"/g) { $tmp = $1; if ($tmp =~ /\x3b/) { print $tmp; }} ' emerging-all.rules |wc -l
58
Note that the 'while (/content:"(.*?)"/g)' loop is necessary above in order to parse all content fields from each Snort rule - using something like 'if (/content:"(.*?)"/' would just parse the very first content field in each Snort rule. Here is an example content field from the "ET MALWARE Vaccine-program.co.kr Related Spyware Checkin" signature: |0d 0a|User-Agent\: Mozilla/3.0 (compatible\; Indy Library)|0d 0a| By contrast, I've seen a few Sourcefire VRT rule sets, and none of them appear to use escaped semicolons in any of their signatures. They always prefer to use the "|3B|" hex notation.

Now, why is this important for fwsnort? The reason is that the current version - fwsnort-1.0.5 - does not properly parse content fields with escaped semicolons. However, this will be corrected in the upcoming fwsnort-1.0.6 release, which will be completed within the next two days or so. In the meantime, here is a link to fwsnort-1.0.6-pre4 that corrects this issue.
Next »