Re: firewall stress testing tool

["Dave Diehl" <thogogorog () gmail com>]

My experience as a network manager and later a firewall developer is that
the only "stress testing tool" that can tell you much about how a system
will hold up under load is something designed to mimic that load as closely
as possible.  How much traffic are you expecting?  How much data per session
on average, and with what distribution?  Spread out over how much time?
Evenly distributed or in surges?  Do you expect to see a large number of
idling persistent connections?  What kind of variety and distribution of
source and destination addresses will be involved?  All of the issues I
mention here and more will have an effect on the performance of even a
trivial stateful inspection firewall (and I didn't even start on the
charateristics of the undesired traffic).  The number of relevant variables
goes up quickly when you instead consider a firewall that does something
more than let you check the "firewall" box in your annual audit.

Of course, as Marcus implies, since the firewall you describe isn't really
doing much of anything, it has a pretty good chance of being stable under
load.

-dave

On 5/19/06, Marcus J. Ranum <mjr () ranum com> wrote:
> pavan shah wrote:
> > I have configured windows 2003 server to allow only traffic to port
> > 80.I want to check for the stability of the firewall under heavy load.
> > Could any one suggest any firewall stress testing tool?
>
> There aren't any decent firewall stress testing tools out there.
> Obviously,
> real network traffic would be the ideal test-bed. Second to that would
> be replays of packets captured at a real firewall installation.
>
> Using something like a smartbits is pointless because they're generating
> synthetic traffic, which would make the firewalls that do any layer 7
> processing look worse (from a performance standpoint) than the firewalls
> that are doing only "stateful inspection" or "deep inspection"  We saw
> a lot of cooked benchmarks early in the IDS days where unscrupulous
> vendors posted unrealistically high performance numbers for IDS packet
> capture by using synthetic traffic that the IDS "knew" to discard. There
> was the famous intrusion.com benchmark done by Meir Communications
> in which intrusion.com demonstrated gigabit speed IDS with no packet
> loss - as long as you threw 1 gb/s of 100K packets at TCP port 0. If
> you have a firewall that (for example) is trying to do protocol state
> parsing for SMTP, it'll look much worse under a synthetic test than
> one that simply goes "wow, that's port 25! let it through if you see a
> HELO!" Under synthetic testing a "stateful" firewall will fare extremely
> well, from a performance standpoint, if all the packets are directed at
> an un-established flow.
>
> One of the ironies of "stress testing" security products is that the
> ones that do LESS security processing almost always do better
> under a load test. Furthermore, the ones that do LESS processing
> appear to do better in terms of (let's loosely call it)"reliability" since
> they will favor letting things through. I saw this back in 1995 when
> one of my customers chose a "stateful" firewall over a proxy because
> in synthetic testing the proxy kept terminating a non-standards-compliant
> FTP command stream, whereas the "stateful" firewall was just looking
> for "PORT" commands and letting everything else through. The
> customer felt that the "stateful" firewall was "better" because it was
> "more reliable" - meaning "easier to get through." So, if you're stress
> testing for "reliability" you need to ask yourself, first, what exactly
> you mean by "reliable." It keeps coming back to the eternal trade-off
> between performance and accessibility on one hand versus conservative
> design and security on the other.
>
> mjr.
>
> _______________________________________________
> firewall-wizards mailing list
> firewall-wizards () listserv icsalabs com
> https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
_______________________________________________
firewall-wizards mailing list
firewall-wizards () listserv icsalabs com
https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards