Re: Evaluating Firewall

[Carson Gaspar <carson () taltos org>]

If you are going to do performance testing as part of the evaluation, here 
are some criteria to look at, based on my experience of vendor benchmark 
... ummm ... "optimization". It was educational to watch the vendors squirm 
in the last RFP I did, when they were forced to report numbers based on 
these criteria. It's amazing how different they were than the initial 
numbers we were given... ;-)

- TCP Connections / sec vs. number of rules

If any state is being kept, only the initial packet / connection traverses 
the rule base. TCP has more state setup work than UDP, in many 
implementations.

Make sure the matching rule is _last_. If there is rule base optimization 
going on, you have to be very careful about your test conditions. So don't 
let it be 10,000 UDP rules followed by a TCP rule, if it branches on 
protocol.

- TCP Packets / sec vs. packet size

This will illuminate the packet rate limitations, as well as the bit rate 
limitations (which are frequently 2 different limits - firewalls rarely can 
keep up at their bit rate limit with 64 byte packets)

Make sure _real_ packets are being used (so TCP sequence numbers are being 
incremented properly, etc.), and that they are being passed - not dropped 
to to rule set or overloading. These numbers should be for 0% packet loss.

- TCP Packets / sec vs. # of established connections (same caveats as above)

Most firewalls have to do a connection lookup for established sessions. 
Good ones will do so with some algorithm that is O(log n) (or so) instead 
of O(n).

- Behavior on saturation

How does the firewall behave once you've gone beyond its capacity? Does it 
gracefully degrade, or fall off a cliff? Do existing connections or old 
connections get priority?

--
Carson Gaspar



_______________________________________________
firewall-wizards mailing list
firewall-wizards () honor icsalabs com
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards