Re: IDS\IPS that can handle one Gig

["Ed Gibbs" <ed () digitalconclave com>]

Tom,

Thanks for the great feedback, and I agree with your comments, except where 
you state that IntruShield and UnityOne slow down to a mediocre dribble. 
That has not been the case with either product, in fact, both have been 
deployed in very large scale environments with the majority of signatures 
and features enabled.

I'm not sure where the "mediocre dribble" is coming from and would hope that 
it's not from your affialiation and bias towards TopLayer.  I can't comment 
on the TopLayer solution since we've  never run into them in any major 
accounts.  By the time these clients analyze the IPS marketplace and narrow 
their product selection, it's typically IntruShield and TippingPoint, and 
having used both, are excellent products.

Ed


----- Original Message ----- 
From: <THolman () toplayer com>
To: <PPalmer () iss net>; <ed () digitalconclave com>; <THolman () toplayer com>; 
<prashant () juniper net>; <focus-ids () securityfocus com>
Sent: Friday, June 03, 2005 5:25 AM
Subject: RE: IDS\IPS that can handle one Gig


> A completely agnostic view follows - there are some important points that
> people are missing out when they're throwing buns at each other...  ;)
>
> 1)  Gigabit performance is irrelevant; it's the packets per second that
> count.  Vendors cheat and claim 1Gb performance based on large packet 
> sizes
> (not real world), or just add up the sizes of all their interfaces.
>
> 2)  In PC architecture, the PCI bus is the bottleneck, not the processor.
>
> 3)  An Intel processor has a large instruction set designed for
> workstation/server performance and GUI operations, and not for packet
> processing.
>
> 4)  An ASIC has a tiny instruction set in comparison, designed for a
> specific task.  So, a 3.2Ghz Intel processor forwarding/processing network
> traffic is on a par with a 133Mhz ASIC designed to do the same thing.
>
> 5)  Processors can only do one thing at once.  Thus, a networking device
> with several processors installed in parallel (ASICs OR Intel) is far more
> effective than a box with a single/dual processor.
>
> 6)  Hard disks do not slow down performance.  They lower reliability as 
> fail
> all the time (!).  RAID would help, but I don't think any security vendor
> offers a RAID array as an integral part of their appliance, so cut to the
> chase, get the HDD off the inline unit and place on a separate management
> machine so we have a reliable distributed architecture that isn't put at
> risk by HDD failure.  On the same note, dual fans and power supplies also
> need to be considered.
>
> 7)  Single-processor machines can easily FORWARD 64-byte packets at
> 'multi-Gig' speeds.  They can do this as the processor doesn't have to do
> anything with them.  As soon as you add intensive operations to the 
> packets
> in question, bearing in mind there is only a single CPU that can only do 
> one
> thing at once, you introduce LATENCY plus reduce pps performance
> DRASTICALLY.  This is where a parallel processing architecture comes into
> it's own and takes leaps forward over what a single-CPU box can do.
>
> In conclusion:
>
> A box with one or two ASICs in is easily outperformed by a PC with the
> latest Intel processor, fast network cards and a good chunk of memory.
> However, the PC is more prone to hard disk failure, which is why you 
> should
> never put one inline if uptime is critical.
>
> A box with several ASICs in will outperform ANY PC-based solution, and ANY
> ASIC solution that relies only on one or two processors.
>
> ..and one comment to Ed with respect to McAfee/TippingPoint
>
> > both products really don't care if you have every signature and then some
> > on.
>
> Yes they do.  If you turn on every signature check with these IPS's, pps
> performance slows to a mediocre dribble...
>
> Inline devices should NOT rely on REGEX signatures - by nature, string
> searching is very resource intensive and best left to a nice fast offline
> IDS running on an up-to-date PC platform, where latency is not going to be
> an issue...
>
> Hope this helps - this isn't an all out war ASIC-based vs PC-based, it's a
> question of architecture and suitability for the job in hand!
>
> Cheers,
>
> Tim
>
>
> -----Original Message-----
> From: Palmer, Paul (ISSAtlanta) [mailto:PPalmer () iss net]
> Sent: 03 June 2005 03:50
> To: Ed Gibbs; THolman () toplayer com; prashant () juniper net;
> focus-ids () securityfocus com
> Subject: RE: IDS\IPS that can handle one Gig
>
> Ed,
>
> I cannot speak to the example you make with firewalls as I have very
> little practical experience in that area. However, I do have
> considerable practical experience with IPS's and I can confidently say
> that the presence (or absence) of ASIC/FPGA technology in a product
> actually implies very little about its true performance. For instance, I
> have a "full" gig switch in my lab from a very respected vendor that try
> as we might we cannot push more than 600Mb/s through its ports. Yet, we
> have COTS PCs that can forward 64-byte packets at multi-gig speeds up to
> the limit of the NIC. That is, the PC architecture is not the
> bottleneck, it is the ASIC on the NIC!
>
> I think you place too much faith in ASICs and FPGAs and grossly
> underestimate the amount of horsepower and throughput available in the
> modern PC architecture. You can use both technologies to achieve very
> high throughputs. You can also use both technologies to produce mediocre
> throughputs.
>
> ASIC/FPGA technology does not preclude the use of a hard drive. Some of
> the IPS's with ASICs in them have hard drives, some do not. To the best
> of my knowledge, all of them, hard drive or not, have non-volatile
> storage that contains sensitive information, so I just do not see the
> merit in the belief that a lack of hard drive somehow confers increased
> security.
>
> Your conjecture that Intrushield and Unity One can outperform anything
> built on a PC to date is wrong. This was almost certainly true when
> those products were first introduced. However, it is no longer true.
> What I see is that the two technologies are fairly closely matched. One
> technology will temporarily edge ahead for a while until the next
> generation of the other technology becomes available.
>
> Again your conjecture that "both products really don't care if you have
> every signature and then some on" is also quite simply wrong. This is
> fairly straightforward to verify through testing.
>
> Paul
>
> -----Original Message-----
> From: Ed Gibbs [mailto:ed () digitalconclave com]
> Sent: Wednesday, June 01, 2005 6:23 PM
> To: Palmer, Paul (ISSAtlanta); THolman () toplayer com;
> prashant () juniper net; focus-ids () securityfocus com
> Subject: Re: IDS\IPS that can handle one Gig
>
>
> Paul,
>
> It has been proven over and over again that networking platforms built
> on
> the PC architecture does not perform equally to a ASIC/FPGA platform.
> Netscreen Firewall was a great example of how a ASIC/FPGA product could
> outperform anything Check Point could provide on Intel (including the
> Nokia/Check Point PC appliance!), especially with 64-byte UDP packets.
> IMHO, anyone placing a security device built around the PC architecture
> "in-line" is asking for trouble.  Would you replace your purpose-built
> Cisco
> routers with PCs running Linux/Zebra?  Of course not.  Do you want an
> appliance with a hard-drive "in-line" on your network.  No again.  What
> happens when the H/D crashes, or in the case of financial/government
> entities, what if the appliance is physically stolen and
> configuration/alerts/etc, are on that H/D?  That's happened.
>
> McAfee IntruShield and TippingPoint UnityOne so far have proven
> performance
> in gig environments.  Both products are built using ASIC/FPGAs and can
> outperform anything built on a PC to date.  There's no compromising by
> disabling signatures to gain performance - both products really don't
> care
> if you have every signature and then some on.
>
> -Ed
>
>
>
>
> ----- Original Message ----- 
> From: "Palmer, Paul (ISSAtlanta)" <PPalmer () iss net>
> To: <THolman () toplayer com>; <prashant () juniper net>;
> <focus-ids () securityfocus com>
> Sent: Wednesday, June 01, 2005 9:20 AM
> Subject: RE: IDS\IPS that can handle one Gig
>
>
> Tim Holman states:
>
> > Agreed - with a system based around PCI / Intel architecture (eg
> > Netscreen IDP, Check Point Interspect/Smart Defense, Cisco 4200, ISS
> > Proventia to name but a few), then it makes sense to turn off various
> > checks to improve performance, but at what cost to security?
>
> This is not a valid conclusion. Whether or not you see performance gains
> by disabling checks does not correlate with the chipsets used. Some of
> the products you mentioned show consistent performance regardless of
> which checks have been enabled. In contrast, some of the "ASIC"
> technology products DO show significant performance differences
> depending on which checks are enabled.
>
> Anyone making a decision based solely upon the perceived advantages of
> the advertised technology of the product is likely to be disappointed.
>
> Paul
>
> -----Original Message-----
> From: THolman () toplayer com [mailto:THolman () toplayer com]
> Sent: Tuesday, May 31, 2005 6:54 PM
> To: prashant () juniper net; focus-ids () securityfocus com
> Subject: RE: IDS\IPS that can handle one Gig
>
>
> Hi Prashant,
>
> Agreed - with a system based around PCI / Intel architecture (eg
> Netscreen IDP, Check Point Interspect/Smart Defense, Cisco 4200, ISS
> Proventia to name but a few), then it makes sense to turn off various
> checks to improve performance, but at what cost to security?
>
> Is it acceptable to turn off vital security features just because the
> shiny new IPS system that you've just bought cannot handle doing too
> many things at once?
>
> Of course not!  ...and to be completely brutal, anyone reading this who
> comes across such a situation should send this equipment back to the
> reseller as being unfit for purpose.  There are plenty of network IPS's
> that are designed to do the job in hand with built-in ASIC technology
> (eg McAfee, TippingPoint and TopLayer) and offer far more punch for the
> money.
>
> There are a whole realm of attacks specifically designed to evade
> IDS/IPS devices through use of fragments.  The theory being that with
> fragmented traffic, an attack can spread itself across multiple packets,
> which all get past string search engines that are looking for a complete
> string, rather than bits of it.
>
> With an IDS, this isn't a problem - the IDS can sit to one side, observe
> the packets coming in, take note once it has seen a stream of fragments
> and reassembled them, and quite happily spend a couple of seconds
> catching up with other stuff before it sends alerts about any signature
> matches it finds in both normal and reassembled traffic.
>
> However, with an IPS, you're supposed to be analysing network traffic at
> line speeds, and you do not have the luxury of hanging around whilst a
> machine designed for client/server purposes works out whether or not
> there's an attack concealed within fragments.  After all, most
> fragmented traffic is genuine traffic - you need to let it through.
>
> Fragmented traffic is a real security threat that needs addressing, and
> disabling security measures that take steps to reassemble and verify
> such traffic will cause a failure of just about any security audit you
> throw at your network, plus leave you open to litigation if your failure
> to address such attacks causes a 3rd party loss.
>
> Regards,
>
> Tim
>
>
> -----Original Message-----
> From: Prashant Khandelwal [mailto:prashant () juniper net]
> Sent: 30 May 2005 06:03
> To: focus-ids () securityfocus com
> Subject: RE: IDS\IPS that can handle one Gig
>
> Adding to this conversation one relevant point would be, Policies which
> are pushed on the sensor makes big difference in the performance of the
> box.
>
> E.g.: If Fragmentation and reassembly turned off it can be observed that
> box performs better as it does not need to take care of tiny fragmented
> packets (In real life having such policies doesn't make any sense).
>
> Over all One should know the Claimed performance figures with avg packet
> size ,What type of traffic was used for achieving that particular
> performance figure ,What kind of policies were pushed on the sensor.
> This can really help to know how a particular IPS can fit in your
> network environment.
>
>
> My 2 cents
> Cheers
> Prashant
>
>
> -----Original Message-----
> From: THolman () toplayer com [mailto:THolman () toplayer com]
> Sent: Thursday, May 26, 2005 2:17 PM
> To: focus-ids () securityfocus com
> Subject: RE: IDS\IPS that can handle one Gig
>
> Hi Randall,
>
> Throughput is unimportant when it comes to choosing an IDS/IPS, and to
> be honest, a bit of a bun fight when you place two vendors side by side
> and start scouring their datasheets for practical information.
>
> What is important, however, is the number of packets per second the
> device can process, the maximum number of connections that such a device
> keeps state for, and last but not least, the latency that such a device
> will introduce into your network if placed inline.
>
> The smaller the packets used in a test, the smaller the performance in
> terms of megabits.  The larger the packets, the bigger the performance
> in terms of megabits.  Unreliable, and totally abused by most vendors on
> their datasheets.  It's quite easy to say 'we support 1000 Mbps', only
> to say in small print the average packet size is 595 bytes.  You only
> need to search Google for '1000 Mbps 595 bytes' and you'll soon find out
> what I mean..
> ;)
>
> The vendor in question, although claiming Gigabit performance, can only
> setup TCP connections at a rate of 5,000 per second - if you do the
> math, you'll soon find out that this represents less that TEN MEGABITS
> per second in 'throughput' terms.
>
> Is it ethical to claim Gigabit performance, only for the potential end
> user to run a number of tests with small packets sizes and find out this
> is not the case?
>
> The moral of the plot is to never trust a datasheet - either thoroughly
> test the products before purchase, or look toward an independent testing
> house, such as NSS (www.nss.co.uk), whom have the resources and
> experience to regularly generate test results that count.
>
> At TopLayer, we regularly deploy into Gigabit environments, and
> encourage the customer to test (using Smartbits, Ixia or Spirent) for
> piece of mind. Rest assured, each time they do this, we pass with flying
> colours, and this is what makes us one of the top market leaders in
> Gigabit IPS solutions.
>
> Regards,
>
> Tim
>
>
> -----Original Message-----
> From: Randall Jarrell [mailto:rgj () msn com]
> Sent: 19 May 2005 16:28
> To: focus-ids () securityfocus com
> Subject: IDS\IPS that can handle one Gig
>
> Greetings,
>
> We are currently evaluating IDS\IPS vendors. We have tried two vendors,
> whom I will not name unless you ask me, that have made claims that they
> can handle a Gig of through put but actually start to fail around the
> 300-500MB range.
>
> Could anyone share a success story of a vendor they are using that is
> handling this type of traffic?
>
> Thanks in advance,
>
> -RGJ
>
> ------------------------------------------------------------------------
> --
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT. Go to
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
>
> to learn more.
> ------------------------------------------------------------------------
> --
>
> ------------------------------------------------------------------------
> --
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT. Go to
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
>
> to learn more.
> ------------------------------------------------------------------------
> --
>
>
> ------------------------------------------------------------------------
> --
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT. Go to
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
>
> to learn more.
> ------------------------------------------------------------------------
> --
>
> ------------------------------------------------------------------------
> --
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT. Go to
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
>
> to learn more.
> ------------------------------------------------------------------------
> --
>
>
> ------------------------------------------------------------------------
> --
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT. Go to
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
> ------------------------------------------------------------------------
> --


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------