Firewall Wizards mailing list archives
Re: your mail
From: tqbf () pobox com
Date: Tue, 30 Jun 1998 03:01:58 -0500 (CDT)
This is incorrect. The methodology also included an extensive evaluation of firewall management and performance.
This discussion is about whether NT Firewalls in general and MS Proxy Server 2.0 in particular are secure. I don't care to discuss how much faster or easier to manage any given firewall is. That isn't my business.
And note that we did put all kinds of caveats telling readers that they *should* think critically about security test results.
A quote from the intro paragraphs of the article: Get a bunch of security gurus together and turn the talk to NT-based firewalls. Then sit back and enjoy the show. "Not ready for the enterprise," they'll sniff. "Not as manageable as Unix utilities." "Not secure." Is that so? Data Communications and National Software Testing Laboratories Inc. (NSTL, Conshohocken, Pa.) just wrapped up an exhaustive test of NT firewalls. And we've got one thing to say to the experts: "Bull." How can we be so sure? We bombarded seven top-selling NT firewalls with nearly 300 forms of attack-without finding any significant security loopholes. (Past tests, including those of Unix products, turned up dozens of flaws.) What's more, these firewalls do an excellent job of locking down potential vulnerabilities in Windows NT itself. Now, I don't consider myself an expert, but I know one thing that definitely IS bull, and that's the idea that you can assess the security of a firewall software platform and compare it to other firewalls by running a network scanner up against it. This is a strong introduction to the article, and nowhere in it is a caveat about the validity of the tests. There is a caveat later ...Our security tests do not in any way certify these firewalls as safe. This evaluation involved the world's foremost experts on these products-the vendors that built them-configuring their own software to withstand well-known attacks in a carefully controlled lab environment. THAT'S A VALID WAY OF DEMONSTRATING THE FIREWALL CODE DOES WHAT IT'S SUPOOSED TO. But it's not the same as saying these firewalls are absolutely secure... [Emphasis mine] Running a network scanner against a firewall is a valid test of the reliability of the code? Exactly why do you think this? The output of a network scanner is a series of automated attacks, designed to assess the vulnerability of an end-system. Network scanners do not emit attacks that are designed to evade firewalls (with a very few exceptions); your test didn't attempt 300 different firewall attacks (as your introduction claims), but rather 300 plain-vanilla clusters of network traffic that any packet filter 3 years ago could have handled.
From the article, it seems that this is the extent of your security
testing methodology, which is what I mean from now on when I use the term "methodology". If this is the case, then your claims about the security of NT firewalls are based solely on the output of a network security scanner that was not designed to assess firewall security. In case you're wondering, I don't know of ANY network security scanner that really is designed to assess firewalls. Your caveats about the validity of your methodology involve stating that "in the real world, new attacks are discovered and firewalls are misconfigured, unlike in our test environment" (my paraphrasing). In the real world, many old firewall attacks were left untried by your methodology, because you relied entirely on a scanner that doesn't attempt them. In the lab room, misconfigurations are irrelevant; you're testing the software, not it's deployment. So what are you warning about here? Your test methodology seems to boil down to "throw the most obvious possible attacks at the firewall, and make sure the firewall blocks them." For example, when assessing the ability of firewalls to block "malicious code" (ActiveX and Java applets): To see if products could block bad code, we set up a Web server with pages containing ActiveX and Java applets. Vendors were asked to configure their firewalls to deny access to these applets. Five could... This is akin to saying "my firewall can block telnet connections, because when I run "telnet" to connect to a protected address, it doesn't work." So what? Attacks against properly configured firewalls don't involve running normal networking commands --- they involve creating pathologically complex streams of network traffic designed to confuse the firewall. Likewise, an informed attack against hosts behind a firewall that utilized ActiveX probably wouldn't involve simply sticking an applet on a web page; rather, it would involve trying to fool the vulnerable browser into running an ActiveX control as a result of a web page that looked nothing like an ActiveX control. Finding problems like this is the essence of firewall security testing. What you did here was not a firewall security test; it was a sanity check. Surprise, you found that the firewalls you tested were sane. One would assume, as you point out in your article, that vendors would have the good sense to spend a few hundred dollars on a scanner to make sure that their firewall performed well in magazine security reviews. What does this say about the actual security of a firewall? Almost nothing. For an article that starts out with the premise that the firewall "experts" of full of BS about the security of NT firewalls, your factual support is awfully weak. ----------------------------------------------------------------------------- Thomas H. Ptacek SNI Labs, Network Associates, Inc. ----------------------------------------------------------------------------- http://www.pobox.com/~tqbf "If you're so special, why aren't you dead?" DISCLAIMERS: I work for Network Associates. I do not speak for them. Network Associates produces CyberCop Scanner, a competitor of ISS's scanner. I do not believe CyberCop Scanner would be any more appropriate as the basis for a firewall test than ISS's tool is. Network Associates also produces Gauntlet, an application-gateway firewall. I have nothing to do with Gauntlet, but you can color your opinions of my opinions as you see fit.
Current thread:
- [no subject] David Newman (Jun 29)
- Re: your mail tqbf (Jun 30)