cipherdyne.org

Michael Rash, Security Researcher



2008 Blog Archive    [Summary View]

Hakin9 Article - Advanced SPA with fwknop

Hakin9 fwknop article In the latest issue of Hakin9 Magazine, I had an article published entitled Advanced SPA with fwknop. It was the goal of this article to introduce the port forwarding capabilities of fwknop that make it possible to reach internal services with automatically generated NAT rules, and also to show how fwknop SPA packets (prior to the 1.9.6 release) could be detected with some well-crafted Snort rules that look for certain encryption and encoding artifacts. Also, with the addition of source IP addresses to SPA digest tracking, it is possible to get a sense of routing paths that might at one time have had sniffers watching for SPA packets if a replay attack against the same fwknopd instance is detected at some later time.

The Snort rules mentioned in the article - updated to take into account the more recent 1.9.6 release - are displayed below. The first Snort rule is designed to look for UDP packets over port 62201 that end with two '=' characters - a potential marker of base64-encoded data (when the original data size was not a multiple of four). The second rule looks for the base64-encoded version of the string Salted__, which is added by the Crypt::CBC module to maintain compatibility with how the OpenSSL library encrypts data. The third rule looks for packets that begin with base64-encoded version of the string 0x8502 which is a marker for data encrypted with GnuPG, and also checks to see of the size of the payload is at least 1000 bytes (SPA packets encrypted with GnuPG tend to be larger than those encrypted with Rijndael). Here are the Snort rules: alert udp any any -> any 62201 (msg:"fwknop pre-1.9.6 SPA traffic"; dsize:>150; pcre:"/==$/"; sid:20080001; rev:1;)
alert udp any any -> any 62201 (msg:"fwknop pre-1.9.2 SPA traffic"; content:"U2FsdGVkX1"; depth:10; dsize:>150; sid:20080002; rev:1;)
alert udp any any -> any 62201 (msg:"fwknop GnuPG encrypted pre-1.9.6 SPA traffic"; content:"hQ"; depth:2; dsize:>1000; sid:20080003; rev:1;)
Any recent release of fwknop (greater than 1.9.5) strips out these identifying markers before transmitting SPA packets on the wire, so these rules are no longer effective at detecting fwknop SPA communications. Also, strong port randomization features were added in fwknop-1.9.4, both for the randomization of the SPA packet destination port as well as the port where the actual connection (say, SSH) is made, so UDP port 62201 is not effective either when these features are used.

Finally, here is an excerpt from the conclusion of the article:

   In the continual arms race that is computer security today, having a good understanding of network communications and how to customize an IDS rule set to an emerging protocol is an important skill. Finally, SPA offers a compelling addition to the tools available for effective server defense; I personally sleep more soundly knowing that arbitrary IP addresses around the Internet cannot see that I have an SSH daemon running, and yet I can access it from wherever I like.

Installing fwknop on Debian and Ubuntu

fwknop Debian packages Franck Joncourt has developed Debian packages of fwknop and fwsnort, and he has provided a set of installation instructions. I've tested this procedure on an Ubuntu-8.04 system as well and it worked quite well. Hopefully it won't be long before fwknop and fwsnort packages make it into the official Debian repository.

To get the latest packages from dthconnex repository, you can add the following lines to your /etc/apt/sources.list file: # cat >> /etc/apt/sources.list
deb http://dthrepo.podzone.org/debian/ sid main
deb-src http://dthrepo.podzone.org/debian/ sid main
Now, the following command will inform you that the public key related to the dthconnex repository (a dedicated GPG key used for signing all packages) is not available: # apt-get update To make it easy for users, a package known as debian-dthconnex-keyring is available and can be installed with the following command: # apt-get install debian-dthconnex-keyring It will add the dthconnex key to your apt keyring, and you can verify this as follows, and now you can run the apt-get update once again and the key warning message should not appear anymore: # apt-key list
[...]
pub 1024D/888B2140 2007-11-25 [expire: 2008-11-24]
uid DthConnex Repository (Unofficial Debian Repository) <debrepository@dthconnex.com>
Now, you can install fwknop (fwknop-server/fwknop-client) or fwsnort safely.

As packages distributed from the dthconnex repository are build from Debian GNU/Linux Sid for amd64/i386 architectures, if you need to backport them or build them for a different architecture, you can get their source like so: # apt-get source fwknop-server

Software Release - fwknop-1.9.7

fwknop-1.9.7 release The 1.9.7 release of fwknop is available for download. This release changes several areas of fwknop related to packaging and packet decoding. First, in order to better support the work of Franck Joncourt to package the cipherdyne.org projects for Debian, all perl module dependencies have been moved into the deps/ directory. Second, it is looking like fwknop will eventually be integrated with Fedora thanks to the work of Mirek Trmac who contributed significant patches (including the removal of the NetPacket dependency). Also, as mentioned in the latest releases of fwsnort and psad, every project release is now signed with a new GnuPG key that is dedicated just for this purpose, and this key can be downloaded here.

The complete ChangeLog for fwknop-1.9.7 appears below:

  • Mirek Trmac from Red Hat contributed several patches so that fwknop can be bundled within the Fedora Linux distribution. These patches implemented the following changes:
    • Updates to fwknopd to remove the NetPacket module as a dependency (this is a particularly important update since it assists with getting fwknop bundled with Debian as well). The patch manually decodes the network and transport layer headers.
    • A patch to make the fwknop init script not start fwknopd by default on Red Hat systems. This patch also supports Fedora init script conventions better (i.e. fwknop instead of the fwknopd name for the lock file in /var/lock/subsys).
    • Updated the fwknop Makefile to respect the OPTS variable which is used in the RPM spec file.
    • Bugfix in fwknop_serv to support the variable expansion code from fwknopd. This was important for the TCPSERV_PID_FILE file which is defined as $FWKNOP_RUN_DIR/fwknop_serv.pid.
    • Updated fwknopd to use the Net::Pcap API valid in Net::Pcap-0.14 for the datalink() function (used to detect the datalink layer type).
  • Updated fwknop, fwknopd, and knoptm to import perl modules out of the /usr/lib/fwknop/ directory if it exists. This allows the perl module path to be manipulated via the --Lib-dir command line argument and 'require' statements instead of the old 'use module' strategy.
  • Added module version output for each non-core perl module used by fwknop and fwknopd in --debug mode. This is mostly useful for the test suite to see which versions of the modules are being used.
  • Added the ability to ignore any local GnuPG 'options' file with a new command line argument --gpg-no-options (for the fwknop client) and a new access.conf config variable GPG_NO_OPTIONS (for the fwknopd daemon). This fixes a problem reported by Mike Holzmann where the 'encrypt-to' option in the default options file was causing SPA packets to exceed 1500 bytes when encrypted with a 2048-bit GnuPG key. Also added the MAX_SNIFF_BYTES to the fwknop.conf file and --Max-packet-size to the fwknop command line to alter the default of 1500 bytes if needed (but this shouldn't really be necessary).
  • Bugfix for 'Premature end of base64 data' and 'Premature padding of base64 data' warning messages from MIME::Base64 errors. Now fwknopd applies more rigorous checks for base64 encoded characters, and either of these two messages above will result in the packet data being discarded before it is sent through any decryption function. Mike Holzmann reported this issue.
  • (Test suite) Added --test-system-fwknop to allow any installed version of fwknop to be installed instead of the scripts bundled within the local source distribution.

fwknop Presentation at BARcamp Chicago

fwknop at BARcamp Chicago Scott Beatty recently prepared a talk about fwknop at the 2008 BARcamp gathering in Chicago. It is nice to see fwknop getting some additional publicity, and Scott emphasizes Single Packet Authorization (SPA) mode communications over the legacy port knocking mode (which often seem to be confused as the same thing). Scott's talk is entitled Protect Your SSH Port (and Others) With fwknop, and has a lot of excellent supporting graphics - including screenshots of both the Nmap cross-platform frontend Zenmap and the fwknop Windows UI in action - to help illustrate authentication concepts and fwknop usage.

From the BARcamp Chicago website: A BARcamp is an ad-hoc gathering born from the desire for people to share and learn in an open environment. It is an intense event with discussions, demos and interaction from participants. The name BARcamp was inspired as a complement to FOOcamp, a private tech gathering run by Tim O'Reilly.

Software Release - psad-2.1.4

psad-2.1.4 released The 2.1.4 release of psad is available for download. This release moves all dependencies into a new deps/ directory - including all perl modules, Snort rules files, and the whois client (from Marco d'Itri). This makes for a cleaner integration of psad with the Debian Linux distribution, thanks to Franck Joncourt. There are also a couple of minor bugfixes and feature enhancements according to the ChangeLog entries below. Finally, all cipherdyne.org projects are now signed with a new GnuPG key available here.

  • Restructured perl module paths to make it easy to introduce a "nodeps" distribution of psad that does not contain any perl modules. This allows better integration with systems that already have all necessary modules installed (including the IPTables::ChainMgr and IPTables::Parse modules). The main driver for this work is to make all cipherdyne.org projects easily integrated with distributions based on Debian, and Franck Joncourt has been instrumental in making this process a reality. All perl modules are now placed within the "deps" directory, and the install.pl script checks to see if this directory exists - a separate psad-nodeps-<ver> tarball will be distributed without this directory. The Debian package for psad can then reference the -nodeps tarball, and a new "psad-nodeps.spec" file has been added to build an RPM from the psad sources that does not install any perl modules.
  • Updated to use the normal system whois client if the /usr/bin/whois_psad path does not exist, and moved the whois/ directory into the deps/ directory. This removes /usr/bin/whois_psad as a strict dependency.
  • Bugfix to honor the IPT_SYSLOG_FILE variable in --Analyze-msgs mode.
  • Switched from the deprecated bleeding-all.rules file to the new emerging-all.rules available from Matt Jonkman at Emerging Threats (http://www.emergingthreats.net).

Software Release - fwsnort-1.0.5

fwsnort-1.0.5 release The 1.0.5 release of fwsnort is available for download. For this release all Snort rules and perl modules packaged within the fwsnort sources have been moved into the deps/ directory so that it is clear which components are dependencies, and this change makes it easy to build "nodeps" tar archives and RPMs of fwsnort that do not include any dependencies. This change was suggested by Franck Joncourt in support of his efforts to create Debian packages of fwsnort, and this will also mean that fwsnort will integrate easily into Linux distributions such as Ubuntu). Also, from now on, all cipherdyne.org project will be signed with a new GnuPG key that is dedicated just for this purpose, and this key can be downloaded here.

The complete ChangeLog for fwsnort-1.0.5 appears below:

  • Replaced the bleeding-all.rules file with the emerging-all.rules file. This is because Matt Jonkman now releases his rule sets at http://www.emergingthreats.net/
  • Restructured perl module paths to make it easy to introduce a "nodeps" distribution of fwsnort that does not contain any perl modules. This allows better integration with systems that already have all necessary modules installed (including the IPTables::ChainMgr and IPTables::Parse modules). The main driver for this work is to make all cipherdyne.org projects easily integrated with distributions based on Debian, and Franck Joncourt has been instrumental in making this process a reality. All perl modules are now placed within the "deps" directory, and the install.pl script checks to see if this directory exists - a separate fwsnort-nodeps-<ver> tarball will be distributed without this directory. The Debian package for fwsnort can then reference the -nodeps tarball, and a new "fwsnort-nodeps.spec" file has been added to build an RPM from the fwsnort sources that does not install any perl modules.
  • Updated to import perl modules from /usr/lib/fwsnort, but only if this path actually exists in the filesystem. This is similar to the strategy implemented by psad. A new variable FWSNORT_LIBS_DIR was added to the fwsnort.conf to support this.
  • Added support for multiple Snort rule directories as a comma-separated list for the argument to --snort-rdir.
  • Moved 'threshold' to the unsupported list since there will be several signatures that use this feature to detect the Dan Kaminsky DNS attack, and fwsnort does not yet support the usage of the iptables --limit match.

NAT and Checking for DNS Cache Poisoning Exploitability

The Dan Kaminsky DNS Checker There are several online DNS checker services that are designed to test nameservers to see if they are vulnerable to the Dan Kaminsky DNS cache poisoning attack. These services perform an important function considering 1) the ease with which vulnerable servers can be exploited, and 2) the critical place DNS has in the Internet infrastructure. If you have not yet checked the nameserver you use for DNS resolution or switched to OpenDNS, let's just say that it is highly recommended, and Dan's talk at the Blackhat Briefings today drove home just how bad the problem is. Dan presented a nice visualization of how much effort has gone into the worldwide patching effort so far, but there is still a lot of work to do.

Online DNS checker services are handy, but unfortunately, not all of these DNS services work the same, and this blog post shows that certain flavors of NAT can make some of these services report nameservers as vulnerable when they really aren't.

Spoiler: The Doxpara and DNS-OARC services handle the NAT case properly by testing against multiple upstream nameservers, whereas the Metasploit test does not yet handle this (although I'm sure it won't be long before this is taken care of).

One of the main requirements for successfully exploiting a vulnerable nameserver is that the source port the server uses for recursive queries must be predictable by the attacker. It's not too difficult for an attacker to profile a potential target by issuing a series of DNS requests for random hostnames (and therefore not cached) in a domain under the attacker's control. Then, by watching how the targeted nameserver issues requests to the authoritative nameserver (controlled by the attacker), it is easy to check whether the source port seems randomly chosen or not. However, if the attacker is only using one authoritative nameserver, this limits the attacker to asking the question:

Are the source ports for recursive queries from the target to a single authoritative nameserver predictable?

...instead of:

Are the source ports for recursive queries from the target across a range of authoritative nameservers predictable?

(For the remainder of this discussion, we'll assume that a targeted nameserver is behind a NAT device, and all recursive DNS requests traverse this device.) The crux of the problem - from the viewpoint of an attacker - is that NAT devices do not always use the same strategy for translating UDP source ports. If the NAT always leaves the source port intact, then there is no problem, and the attacker only needs one authoritative DNS server in order to see whether the target randomizes source ports. If the NAT imposes an "n+m" rule, then this is also advantageous to the attacker with only one authoritative nameserver - even if the targeted DNS server randomizes source ports itself. But, if the NAT randomizes source ports according to a rule like "for the next 30 seconds, give all UDP packets with source port 12345 destined for IP1 a new source port of 48567, and all packets to IP2 a new source port of 34432, etc.", then the attacker with only one authoritative nameserver would see that all incoming queries have the same source port (at least for 30 seconds). Given that an actual DNS cache poisoning attack requires spoofed responses from nameservers that are (presumably) not under the control of the attacker, such a NAT device would thwart an attack because it would assign a new (random) port for the set of queries to each authoritative nameserver.

The iptables firewall with it's SNAT --random feature implements a strategy similar to the above, and we show how a few DNS checker services handle this for a targeted nameserver 22.2.2.2 (the IP is for illustration purposes only).

First, Dan's DNS checker reports the following: Your name server, at 22.2.2.2, appears to be safe, but make sure the ports
listed below aren't following an obvious pattern (:1001, :1002, :1003, or
:30000, :30020, :30100...). Requests seen for c26cd2dfd938.doxdns5.com:
22.2.2.2:40796 TXID=27431
22.2.2.2:28530 TXID=7761
22.2.2.2:56025 TXID=64688
22.2.2.2:30711 TXID=37206
22.2.2.2:24472 TXID=84
Good.

Now, DNS OARC reports: [attacker]# dig @22.2.2.2 +short porttest.dns-oarc.net TXT
porttest.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.pt.dns-oarc.net.
"22.2.2.2 is GREAT: 26 queries in 2.5 seconds from 26 ports with std dev 18972"
This is also good.

Finally, on the exploit side of things, Metasploit's service reports: [attacker]# ./msfconsole
msf > use auxiliary/spoof/dns/bailiwicked_host
msf auxiliary(bailiwicked_host) > set RHOST 22.2.2.2
RHOST => 22.2.2.2
msf auxiliary(bailiwicked_host) > check
[*] Using the Metasploit service to verify exploitability...
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] FAIL: This server uses a static source port and is vulnerable to poisoning
msf auxiliary(bailiwicked_host) >
But, the above argument shows that the Metasploit result is a false positive. After seeing such a result, the attacker might continue on with Metasploit and create a ton of DNS queries and spoofed responses, only to have the exploit attempt fail because the source port really isn't predictable for DNS queries issued to nameservers outside the attacker's control.

The prevalence of such NAT deployments is most likely low, but it would make an interesting research project to try and determine how many DNS servers are protected by such a mechanism. However, a key detail is that for testing this is that the DNS servers fundamentally must still select the same source port for recursive queries since if not then the iptables SNAT --random option will be more random (even for rapid sets of recursive queries). Source port randomization is exactly one of the things the patches to DNS servers is designed to address, so the opportunity to perform such research is closing. Also, hopefully no one is still setting query-source address * port within the named.conf file.

Update: 08/09/08 Jeff Jarmoc mentioned to me that the implementations of NAT on Cisco and Check Point devices can de-randomize source ports of services running on internal networks, and this problem even affects DNS servers that have been patched. That is an important point, and the problem was also noted in this blog post. The vendors are patching the issue, but there is a lot of work left.

Metasploit DNS Cache Poisoning and iptables Countermeasures

Metasploit and DNS Cache Poisoning Attacks On July 23rd, H D Moore, I)ruid, and the Metasploit Project released an exploit for the Dan Kaminsky DNS cache poisoning attack - the full details of which will be released at the Blackhat Briefings. We know that source port prediction for recursive queries is a key component to successfully poison a nameserver's cache, and the Metasploit exploit code offers the ability to check a targeted nameserver for predictable source ports. This check is implemented by sending a set of TXT queries against the metasploit.com domain to a targeted server, and - if recursion is enabled - the server will in turn send these queries (which are randomized and therefore not cached) to a nameserver that is authoritative for the metasploit.com domain. The information returned by the metasploit.com nameserver contains the source port the targeted nameserver used to issue the queries. If the source port exhibits a high degree of predictability (such as if the query-source address * port named.conf directive is used or if an external NAT device removes randomness in the source port on its own), then it is much easier to spoof responses to queries against the target from other nameservers and increase the chances that one of these spoofed responses will be cached.

Here is an example of using the source port check feature in Metasploit. Some output has been abbreviated, and the IP addresses 11.1.1.1 (hostname: attacker) and 22.2.2.2 (hostname: target) are used only for illustration purposes: [attacker]# ./msfconsole
msf > use auxiliary/spoof/dns/bailiwicked_host
msf auxiliary(bailiwicked_host) > set RHOST 22.2.2.2
RHOST => 22.2.2.2
msf auxiliary(bailiwicked_host) > check
[*] Using the Metasploit service to verify exploitability...
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] FAIL: This server uses a static source port and is vulnerable to poisoning
So, the source port used by the targeted nameserver is always 30001 - convenient.

If we examine the traffic generated by Metasploit on the wire, the TXT queries exhibit some nice structure. Note the spoofprobe-check and .red.metasploit.com strings in both the incoming queries responses (actually, incoming queries contain the bytes |03|red|0a|metasploit|03|com in Snort syntax): [attacker]# tcpdump -i eth0 -l -nn port 53
22:53:40.594082 IP 11.1.1.1.40866 > 22.2.2.2.53: 0+ TXT? spoofprobe-check-1-10913572965.red.metasploit.com. (67)
22:53:40.730809 IP 22.2.2.2.53 > 11.1.1.1.40866: 0 1/1/0 TXT "22.2.2.2:30001 IN IN::TXT spoofprobe-check-1-10913572965.red.metasploit.com" (182)
Further, in the query responses, the string IN::TXT is also returned. So, using this information, we can build iptables rules that leverage the string match extension to inspect application layer data for these strings. We can then have iptables take action such to log or drop packets that match.

If a nameserver is running locally on a Linux system, then the following rules detect inbound requests from the attacker (see the usage of the iptables --hex-string argument to describe the non-printable bytes in the incoming DNS request), as well as responses from the metasploit.com server to outbound recursive requests from the targeted nameserver. Note that you might want to combine the LOG rules with the iptables limit match in order to reduce the number of log messages created during an actual attack. Still, I find that having more data is usually good, and the number of source port reconnaissance queries is much less than the number of spoofed responses when a cache poisoning attempt is made anyway, so it shouldn't be too burdensome to leave off the limit match. [target]# iptables -I INPUT 1 -p udp --dport 53 -m string --string "spoofprobe-check" --algo bm -m string --hex-string "|03|red|0a|metasploit|03|com" --algo bm -j LOG --log-prefix "METASPLOIT DNS RECON QUERY "
[target]# iptables -I INPUT 1 -p udp --dport 53 -m string --string "spoofprobe-check" --algo bm -m string --hex-string "|03|red|0a|metasploit|03|com" --algo bm -j DROP

[target]# iptables -I INPUT 1 -p udp --sport 53 -m string --string "IN::TXT" --algo bm -m string --string "spoofprobe-check" --algo bm -m string --string ".red.metasploit.com" --algo bm -j LOG --log-prefix "METASPLOIT DNS RECON RESP "
[target]# iptables -I INPUT 1 -p udp --sport 53 -m string --string "IN::TXT" --algo bm -m string --string "spoofprobe-check" --algo bm -m string --string ".red.metasploit.com" --algo bm -j DROP
Similar rules can be added to the FORWARD chain (along with specifying the internal subnet or input interface so that directionality can be established) for nameservers that are deployed on a separate system behind a Linux system running iptables. The most important rules above are probably the first two, since matching packets reveal the source IP of the attacker. However, this is of limited use because the actual cache poisoning attack will involve packets spoofed from other authoritative nameservers, and it is possible to collect source port information from other sources. Still, having information about someone doing source port predictability reconnaissance against one of your nameservers with Metasploit is worth knowing.

With the rules above in place, the 'check' step in Metasploit is unable to tell that the targeted nameserver even responds to recursive queries at all, and back on the firewall system several METASPLOIT DNS RECON QUERY log messages are written to syslog by iptables: msf auxiliary(bailiwicked_host) > check
[*] Using the Metasploit service to verify exploitability...
[*] ERROR: This server is not replying to recursive requests

[target]# tail /var/log/messages
Aug 2 06:39:55 target kernel: [933142.545502] METASPLOIT DNS RECON QUERY IN=eth0 OUT= MAC=00:13:46:3b:41:4c:00:12:46:c2:60:44:09:00 SRC=11.1.1.1 DST=22.2.2.2 LEN=96 TOS=0x00 PREC=0x00 TTL=63 ID=33573 DF PROTO=UDP SPT=40273 DPT=53 LEN=76
Aug 2 06:39:55 target kernel: [933142.637446] METASPLOIT DNS RECON QUERY IN=eth0 OUT= MAC=00:13:46:3b:41:4c:00:12:46:c2:60:44:09:00 SRC=11.1.1.1 DST=22.2.2.2 LEN=96 TOS=0x00 PREC=0x00 TTL=63 ID=33574 DF PROTO=UDP SPT=40273 DPT=53 LEN=76
On another note, before using Metasploit to test your DNS infrastructure to see if it is exploitable, it is important to know whether your local network allows spoofed packets out. Many firewalls can be configured to drop spoofed packets from internal systems, and even my little LinkSys router does this. Because there is no mechanism in Metasploit currently (as far as I know) to detect whether your local network filters spoofed packets (and building such a mechanism would be tricky for various technical reasons), an incorrect assumption can result in a cache poisoning attack that has no possibility of succeeding but that also generates thousands of DNS queries at the same time.

Finally, for those unpatched nameservers running behind an iptables firewall, the SNAT --random option can provide a work-around for predictable source ports. Nevertheless, the emphasis should always be on patching vulnerable servers since the source port problem is only one aspect of the vulnerability in DNS.

The Last HOPE Conference, Port Knocking and SPA Talk Wrap-up

SPA talk slides from the Last HOPE conference This past weekend at the Last HOPE conference in NYC I gave a talk entitled "Port Knocking and Single Packet Authorization: Practical Deployments" (slides). The talk was fairly well attended - I estimate about 100 people or so - and there were several good questions from the audience. Some of the questions deserved a more thorough answer than I was able to provide during the talk, so this blog post fleshes out some of my original answers. If you attended the talk and would like to see more discussion of one of these questions, or if you just have a question about fwknop in general, please email me.

Q: Does fwknop run on the iPhone?
A: Given that the iPhone runs Mac OS X as well as perl, it should be feasible to run fwknop on the iPhone at least as a client. If ipfw is also available for the iPhone and AT&T does not filter inbound connections to servers that are running on an iPhone (I don't personally have one so I have not verified this), then it should also be possible to run the fwknopd daemon to sniff SPA packets.

Q: Why not build a covert channel over bits in the TCP header? Wouldn't this provide a way to implement a more simplistic userspace daemon for parsing inbound traffic?
A: There are certainly a decent number of bits available within the TCP header (and IP header too) that could be used to encode information within a covert channel, but my concern would be that not enough bits would be available to accomplish everything necessary to provide good security. Making such packets non-replayable while offering an encrypted payload at the same time I think would be difficult. That said however, one could envision a hybrid approach where bits are used first within packet headers to "get to the next stage", and this would allow the first stage to be implemented by a simplistic piece of code (such as parsing a firewall log). (This was suggested to me by Jacob Appelbaum at the end of the talk.) Still, there would be a permissions problem on the client side because tweaking bits at this level would mean that one cannot just open a socket and send data - a raw socket would be required.

Q: Have you audited the perl parser code or associated perl modules for security vulnerabilities?
A: Yow, this one was a zinger, and gets to the heart of the code complexity problem. I certainly cannot claim that all of the code used by fwknop is completely secure (no one ever can really for any piece of code), but at least attacking the fwknopd daemon is not as easy as just scanning for some vulnerable piece of software that advertises itself via a TCP socket. I'm working on re-writing fwknop in C though to make it much more lightweight and less complex, and this should help to address concerns about the complexity of perl modules while also allowing fwknopd to run on embedded distributions such as OpenWRT.

Jacob Appelbaum attended the talk, and he asked the later two questions above (which were excellent and thought provoking). Later on Sunday, I spoke briefly with Jacob and during that conversation he suggested sending SPA packets over DNS queries in order to once again return to a log parsing model for SPA instead of requiring a sniffer, and using DNS would also imply compatibility with Tor because it already handles DNS queries. I think this is a great idea, and I have added it to the fwknop TODO list - it will definitely be implemented as an option for fwknop communications.

On another note, Jacob gave a panel discussion on Saturday of the recent Debian OpenSSL vulnerability along with Dino Dai Zovi and Karsten Nohl. Hopefully the video of this talk will be posted online - it was quite excellent and provides perspective on just how damaging a seemingly innocuous and helpful "fix" can have far-reaching implications for computer (in)security. In this case, a Debian developer noticed that valgrind detected the usage of uninitialized memory by the OpenSSL library, and interpreted this as a programming bug that needed to be fixed. It turns out that OpenSSL actually uses uninitialized memory as a way to gather additional entropy during the key generation process, and removing this code had the effect of reducing the security of all keys generated by any application on Debian that used OpenSSL, which is a huge list of applications. We are going to feel the effects of this bug for a long time to come even if every vulnerable system has already been patched because there are a lot of weak keys still in use.

Software Release - fwknop-1.9.6

software release fwknop-1.9.6 The 1.9.6 release of fwknop is ready for download. This release was made at The Last HOPE conference over the weekend in NYC, and introduces several new features that make SPA traffic more difficult to detect on a network even when an IDS is watching. Previous to the 1.9.6 version of fwknop, it was possible to write Snort rules to detect SPA traffic by looking for invariant artifacts from base-64 encoding and also from encryption operations (both Rijndael and GnuPG). These artifacts are now stripped out by the fwknop client before being transmitted on the wire, and the result is that SPA packets are now more highly randomized. This implies that SPA packets generated by the 1.9.6 release are not compatible with older fwknop deployments by default, but the client does offer command line arguments to maintain compatibility if necessary.

Here is an excerpt from the ChangeLog:

  • SPA packets are base64-encoded by the fwknop client, and this encoding pads data with '=' chars until the total length of the encoded data is a multiple of four. This characteristic can be used within a Snort rule to assist in the detection of SPA communications. The 1.9.6 release of fwknop strips out these padding characters before the client sends an SPA packet, and the fwknopd server adds them back in (to form a multiple of four) before base64 decoding the packet data. This reduces the level of identifying information in SPA packets and therefore makes it more difficult to detect the usage of SPA for service access. For reference, a Snort rule that would detect SPA packets via the trailing '=' chars (previous to this release) would be:

    alert udp any any -> any 62201 (msg:"fwknop SPA traffic"; dsize:>150; pcre:"/==$/"; sid:20080001; rev:1;)

  • According to the 'file' command (via it's 'magic') database, files that are encrypted with GnuPG begin with 0x8502, and this is true for SPA packets generated by fwknop (previous to this release). In fwknop-1.9.6, the "hQ" prefix is removed by the fwknop client and added back in by the fwknopd server if it doesn't exist. This measure is another effort to make SPA packets more difficult to detect on the wire, such as with the following Snort rule:

    alert udp any any -> any 62201 (msg:"fwknop GnuPG encrypted SPA traffic"; content:"hQ"; depth:2; dsize:>1000; sid:20080003; rev:1;)

  • Updated the fwknop client to randomize the UDP source port for default SPA packet generation. There is also a new command line argument --Source-port <port> to allow the user to manually set the source port on the fwknop client command line. A lot more attention is given now to source ports after the Dan Kaminsky DNS caching exploit, and it turns out that even on Linux that the kernel did not randomize UDP source ports until the 2.6.24 kernel. Of course, any userspace process is free to request a random port itself, but if a userspace application did not build this in then it would be up to the kernel to assign a source port. In the case of Linux, here are two links that show the change to the kernel code as well as the ChangeLog entry for UDP source port randomization:

    Kernel commit

    ChangeLog-2.6.24