Re: I don't need no stinking firewall!

[Joel Jaeggli <joelja () bogus com>]

Dobbins, Roland wrote:
> On Jan 9, 2010, at 7:52 AM, Joel Jaeggli wrote:
>
> > see my post in the subject, a reasonably complete performance 
> > report for the device is a useful place to start.
>
> The problem is that one can't trust the stated vendor performance 
> figures, which is why actual testing is required.  I've seen 
> instances in which actual performance is 5% or less of vendor 
> assertions (i.e., vendor constructed a highly artificial scenario in 
> order to be able to make a specific claim which doesn't hold up in 
> real life).

I'll go out on a limb and just quote myself since you didn't fully.

> > if you know what the maximum session rate and state table size for
> > the device are, you have a pretty good idea at what rate of state 
> > instantiation it will break. rather frequently it's more than two 
> > orders of magnitude lower than the peak forwarding rate.

two orders of magnitude lower is 1% of forwarding rate. It could be
lower but it's probably not 3 orders of magnitude.

rfc 3511 testing won't tell you that much that's useful in the real
world. but it will tell you how many concurrent sessions you can
establish (which is almost purely a function of the amount of ram for
that data strcture) through the DUT and how quickly you can establish
them (which may vary with your rule base but will almost certainly never
get better). vendors are mostly honest about that becuase you can
trivially replicate that test even with fairly low end equipment on all
but the biggest stateful devices.

> Also note that most vendors don't perform negative testing, 
> astonishing though that may seem.

> -----------------------------------------------------------------------
>  Roland Dobbins <rdobbins () arbor net> // 
> <http://www.arbornetworks.com>
>
> Injustice is relatively easy to bear; what stings is justice.
>
> -- H.L. Mencken