Re: Tuning false positives

["David W. Goodrum" <dgoodrum () nfr com>]

Turning off detection techniques alltogether is not usually the 
preferred method. I'll give you a few examples of some alerts that NFR 
might generate out of the box and how you would tune them.

Scenario 1: You deploy a sensor inside your network, and out of the box, 
you start to trigger hundreds of false positives because we see lots of 
"public" SNMP community strings. This is quite common inside people's 
networks. At the very least, almost every printer you have will have 
this on by default. You then have a number of options. You can say that 
these alerts are relevant, and that you will make it your mission to 
obliterate public community strings on your network. Or you can accept 
that you can never get rid of these public community strings altogether, 
so you decide that public community strings are "okay" inside your 
network. Just so long as you don't see them in certain areas, such as 
your critical data center, or your DMZ. So, you _tune_ the system. You 
decide to ignore public community strings from every IP address inside 
your network, except the ranges in your DMZ and critical data center. 
This gets rid of most of the noise from userland, but still allows you 
to keep a tight grip on weak community strings on critical or exposed 
equipment.

Scenario 2: Same deployment as above. NFR alerts on weak passwords, such 
as short passwords, alphabetic only passwords, etc. This would apply to 
protocols such as FTP, telnet, HTTP, POP, IMAP, MYSQL, etc (all the 
clear text protocols). The default policy alerts on passwords less than 
8 characters long. The NFR box starts alerting when it sees users 
connecting to FTP servers outside of your network. i.e. You are allowing 
users to connect outbound on port 21 to servers that you don't control, 
which is fine. However, you can't control the password policies on those 
servers, so when the users create 7 character passwords that are all 
alphabetic, we generate 2 alerts that you don't care about. In the 
course of the day, depending on the number of users you have, and the 
type of company you are (for example, a research organization), you 
might generate hundreds of these alerts... maybe even thousands. And you 
don't care about them at all. But, you don't want to just turn off this 
check. So, you _tune_ the system. You decide to ignore these alerts as 
long as the source IP is inside your network, and the dest IP is outside 
your network. This gets rid of the noise, but still alerts you if 
somebody outside the network tries to come in with a weak password, or 
if somebody inside your network connects to another IP address inside 
your network with a weak password.

There are lots of these types of scenarios, and a solution for every one 
that allows you to get rid of noise. Tuning is the right solution. 
Turning off detection techniques is not tuning.

In my experience 5 or 6 alerts generate 90% of the noise. If you have a 
system with 1000's of checks, and you find yourself tuning out only a 
couple of alerts, then you're in good shape. Statistically speaking, 
that's actually really good. In the end you still might end up, after 
tuning, with 4 or 5 alerts that still popup as false positives 
occasionally, that you just can't figure out the best way to tune them. 
So, you classify them as really low priority. If any other alerts shows 
up (out of the 1000's of possible alerts you've never seen before), you 
can quickly identify it without it getting lost in the noise.

Some might recommend using a SIM product to try to manage the large 
volume of alerts. To that, I have one answer: Bad data in... Bad data 
out. SIM products are not a solution for tuning your system. SIM 
products do server a good purpose, but that purpose, IMHO, does not 
replace the intelligence that goes into tuning a system. I've seen too 
many SIM implementations fail because customers want to use them as a 
silver bullet to their data overload issues.

Hope this helps,

dave

--
David W. Goodrum, CEH
(nfr)(security)
http://www.nfr.com
(M)703.731.3765
(O)240.747.3425
(F)240.632.0200



Sam Heshbon wrote:

> My company is testing a few intrusion detection & prevention products. On the first few hours/days
> after deployment the machines alert on ten of thousands of events, which is way too much for us to
> ever go through, most of which are false alarms.
>   
> The vendor’s solution is tuning the systems, which means shutting down signatures, detection
> mechanisms, omitting defragmentation tests and so on. These tunings do reduce dramatically the
> number of alerts, but it seems most of the detection capabilities have been shut off too, so
> things are 
> nice and quite but we've no idea what's really going on in our network apart from catching the
> trivial threats such as old worms, which don’t get false alarms.
> Has anyone encountered this situation? Anyone got a solution?
>   
> Thanks
>   
> Sam
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
>
> ------------------------------------------------------------------------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it 
> with real-world attacks from CORE IMPACT.
> Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
> to learn more.
> ------------------------------------------------------------------------
>
>  


------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
------------------------------------------------------------------------