RE: Signatures taking down network

["Palmer, Paul (ISSAtlanta)" <PPalmer () iss net>]

Sam,

I do not know of one of them that feel that they are being used as
guinea pigs. If they did, I doubt that they would participate. Indeed,
for most this process is complementary to their own QA process. The
content updates are typically deployed on passive sensors so the risk is
acceptably low. The end result is that they can actually deploy security
content updates sooner with more confidence than they could otherwise,
since by the time the security content is released to the general
community they know exactly how it will perform on their network.

Paul


-----Original Message-----
From: Sam Evans [mailto:wintrmte () gmail com] 
Sent: Wednesday, January 18, 2006 3:43 PM
To: Palmer, Paul (ISSAtlanta)
Cc: David Williams; focus-ids () securityfocus com
Subject: Re: Signatures taking down network


Paul,

I'm curious to know how these customers feel about their networks being
used as guinea pigs?

"ISS is also a managed services provider for a large number of
customers. It leverages its Managed Security Services unit to
efficiently field test security content updates prior to shipping these
to the larger customer base. The update is shipped to a preauthorized
group of sensors on various customers' networks in order to see live
traffic in production networks. This added test cycle benefits all
customers, and especially those customers in the test group, as we can
be sure that these customers' traffic is accurately analyzed and
verified. "


On 1/18/06, Palmer, Paul (ISSAtlanta) <PPalmer () iss net> wrote:
> David,
>
> I work for ISS.
>
> I can tell you that it is very challenging for the vendors to produce 
> quality signatures today. Vulnerabilities are announced at a record 
> pace. There is now financial incentive for criminals to research and 
> discover their own 0-day vulnerabilities, so we are going to see a lot

> more of these now. This places incredible pressure on the vendors to 
> produce protection signatures as quickly as possible so as not to 
> leave their customers exposed.
>
> This should not be taken as an excuse, it is just the reality with 
> which each vendor must successfully come to terms.
>
> Every schedule is defined by the scope of the problem, the time 
> available, and the resources available. If you fix any two of them, 
> the other has to give. So, if you will produce a quality signature, 
> you must either invest more time or more resources. Given that the 
> time element isn't very flexible for this problem, it means that you 
> must invest more resources. To do otherwise results in late delivery 
> or poor quality (stability, false negatives, false positives, etc).
>
> So, the vendors that invest the most resources into the problem are 
> the ones that are going to produce the best quality over the long run.

> However, there is a still a problem with scalability in the face of 
> the ever increasing rate that vulnerabilities (and for many vendors, 
> exploits also) are discovered. Only the vendors with very efficient 
> processes are going to be able to stay in the business over the long 
> run.
>
> The QA challenge for IPS products is unlike any other I have 
> experienced. One of the things IPS vendors learn very quickly is that 
> lab traffic alone is insufficient for testing IPS products. The 
> products must be exposed to as much real world traffic prior to the 
> release of updates as possible. The second lesson is that there is a 
> very wide variety of traffic on customer networks. An update exposed 
> only to lab traffic can work flawlessly on 9 out of 10 customer sites 
> and fail miserably on the tenth. Even if it is 99 out of 100, it is 
> unacceptable.
>
> ISS is also a managed services provider for a large number of 
> customers. It leverages its Managed Security Services unit to 
> efficiently field test security content updates prior to shipping 
> these to the larger customer base. The update is shipped to a 
> preauthorized group of sensors on various customers' networks in order

> to see live traffic in production networks. This added test cycle 
> benefits all customers, and especially those customers in the test 
> group, as we can be sure that these customers' traffic is accurately 
> analyzed and verified.
>
> Every vendor is different. The ones that can and do consistently 
> invest in their processes will have the better quality record over the

> long run. Look for the ones that have been investing in quality long 
> enough to have developed mature and efficient processes to have the 
> best quality.
>
> So, finally, I would expect that the trend in the industry overall is 
> towards higher quality as some vendors consistently improve their 
> processes and the ones that do not are gradually winnowed out.
>
> Paul
>
> -----Original Message-----
> From: David Williams [mailto:dwilliamsd () gmail com]
> Sent: Saturday, January 14, 2006 9:04 AM
> To: focus-ids () securityfocus com
> Subject: Signatures taking down network
>
>
> I'm evaluating a Tipping Point box and after gettting the latest 
> signatures I'm having problems with the box "crashing".  My goal is 
> not to bash Tipping Point, but instead to gather information on how 
> often people have seen this type of thing among IPS boxes.
>
> Is there a trend with vendors to roll out signatures as fast as 
> possible without proper QA?  This brings up a lot of questions about 
> deploying IPS.  I want two opposite things from my vendors:  1) I want

> the latest signatures super fast.  2)  I want proper QA so that it 
> doesn't bring down my network.  I realize those two things are 
> contradictory, but I thought I'd throw it out there to see if anybody 
> had any thoughts.
>
> thanks,
>
> d
>
> ----------------------------------------------------------------------
> --
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it
> with real-world attacks from CORE IMPACT.
> Go to 
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
>
> to learn more.
> ----------------------------------------------------------------------
> --
>
>
> ----------------------------------------------------------------------
> --
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it
> with real-world attacks from CORE IMPACT.
> Go to 
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
------------------------------------------------------------------------
>


------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
------------------------------------------------------------------------