RE: Tuning false positives

["Ofer Shezaf" <Ofer.Shezaf () breach com>]

I think that David's discussion is very good and would like to add two
points:

1. Most posts in this thread used the term IDS/IPS. However, IDS and IPS
are very different when it comes to false positives. An IDS, i.e. a
monitoring system, needs to display only events of interest, that a
human operator can and should resolve. An IPS on the other hand can
block traffic which is not malicious but is not interesting. This is
very common with behavioral systems as many operating mistakes can be
rejected, but are not interesting. 

A good example would be mistypes URL, which the web server might reject
anyway. An IPS could safely block the request, while as a reported IDS
event it has no value. Now if this mistype URL occurs due to a mistyped
link to the site, the reported event would flood the monitoring system.
I call these "soft" False positives: correct but meaningless.

2.  As IDSes evolved into monitoring systems, they added a new role -
they do not only detect intrusions, they also monitor the level of
security of the network. I think that David's examples show this very
clearly. As such, some management of the events is due: should an IDS
alert on a weak password each time it is detected? Only on the 1st time?
Every week? Alternatively the system could have a non event driven mode
in which users/computers and information about them is listed.

Therefore I do not agree with David regarding the role of a SIM. A good
SIM could provide some of these aggregations over the raw event data. Of
course, an IDS system could also do that. 

~ Ofer

> -----Original Message-----
> From: David W. Goodrum [mailto:dgoodrum () nfr com]
> Sent: Wednesday, December 28, 2005 4:01 PM
> To: Sam Heshbon
> Cc: focus-ids () lists securityfocus com
> Subject: Re: Tuning false positives
>
> Turning off detection techniques alltogether is not usually the
> preferred method. I'll give you a few examples of some alerts that NFR
> might generate out of the box and how you would tune them.
>
> Scenario 1: You deploy a sensor inside your network, and out of the
box,
> you start to trigger hundreds of false positives because we see lots
of
> "public" SNMP community strings. This is quite common inside people's
> networks. At the very least, almost every printer you have will have
> this on by default. You then have a number of options. You can say
that
> these alerts are relevant, and that you will make it your mission to
> obliterate public community strings on your network. Or you can accept
> that you can never get rid of these public community strings
altogether,
> so you decide that public community strings are "okay" inside your
> network. Just so long as you don't see them in certain areas, such as
> your critical data center, or your DMZ. So, you _tune_ the system. You
> decide to ignore public community strings from every IP address inside
> your network, except the ranges in your DMZ and critical data center.
> This gets rid of most of the noise from userland, but still allows you
> to keep a tight grip on weak community strings on critical or exposed
> equipment.
>
> Scenario 2: Same deployment as above. NFR alerts on weak passwords,
such
> as short passwords, alphabetic only passwords, etc. This would apply
to
> protocols such as FTP, telnet, HTTP, POP, IMAP, MYSQL, etc (all the
> clear text protocols). The default policy alerts on passwords less
than
> 8 characters long. The NFR box starts alerting when it sees users
> connecting to FTP servers outside of your network. i.e. You are
allowing
> users to connect outbound on port 21 to servers that you don't
control,
> which is fine. However, you can't control the password policies on
those
> servers, so when the users create 7 character passwords that are all
> alphabetic, we generate 2 alerts that you don't care about. In the
> course of the day, depending on the number of users you have, and the
> type of company you are (for example, a research organization), you
> might generate hundreds of these alerts... maybe even thousands. And
you
> don't care about them at all. But, you don't want to just turn off
this
> check. So, you _tune_ the system. You decide to ignore these alerts as
> long as the source IP is inside your network, and the dest IP is
outside
> your network. This gets rid of the noise, but still alerts you if
> somebody outside the network tries to come in with a weak password, or
> if somebody inside your network connects to another IP address inside
> your network with a weak password.
>
> There are lots of these types of scenarios, and a solution for every
one
> that allows you to get rid of noise. Tuning is the right solution.
> Turning off detection techniques is not tuning.
>
> In my experience 5 or 6 alerts generate 90% of the noise. If you have
a
> system with 1000's of checks, and you find yourself tuning out only a
> couple of alerts, then you're in good shape. Statistically speaking,
> that's actually really good. In the end you still might end up, after
> tuning, with 4 or 5 alerts that still popup as false positives
> occasionally, that you just can't figure out the best way to tune
them.
> So, you classify them as really low priority. If any other alerts
shows
> up (out of the 1000's of possible alerts you've never seen before),
you
> can quickly identify it without it getting lost in the noise.
>
> Some might recommend using a SIM product to try to manage the large
> volume of alerts. To that, I have one answer: Bad data in... Bad data
> out. SIM products are not a solution for tuning your system. SIM
> products do server a good purpose, but that purpose, IMHO, does not
> replace the intelligence that goes into tuning a system. I've seen too
> many SIM implementations fail because customers want to use them as a
> silver bullet to their data overload issues.
>
> Hope this helps,
>
> dave
>
> --
> David W. Goodrum, CEH
> (nfr)(security)
> http://www.nfr.com
> (M)703.731.3765
> (O)240.747.3425
> (F)240.632.0200
>
>
>
> Sam Heshbon wrote:
>
> > My company is testing a few intrusion detection & prevention
products. On
> the first few hours/days
> > after deployment the machines alert on ten of thousands of events,
which
> is way too much for us to
> > ever go through, most of which are false alarms.
> >
> > The vendor's solution is tuning the systems, which means shutting
down
> signatures, detection
> > mechanisms, omitting defragmentation tests and so on. These tunings
do
> reduce dramatically the
> > number of alerts, but it seems most of the detection capabilities
have
> been shut off too, so
> > things are
> > nice and quite but we've no idea what's really going on in our
network
> apart from catching the
> > trivial threats such as old worms, which don't get false alarms.
> > Has anyone encountered this situation? Anyone got a solution?
> >
> > Thanks
> >
> > Sam
> >
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam protection around
> > http://mail.yahoo.com
>
> -----------------------------------------------------------------------
-
> > Test Your IDS
> >
> > Is your IDS deployed correctly?
> > Find out quickly and easily by testing it
> > with real-world attacks from CORE IMPACT.
> > Go to
http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> > to learn more.
>
> -----------------------------------------------------------------------
-
> >
------------------------------------------------------------------------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it
> with real-world attacks from CORE IMPACT.
> Go to
http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
------------------------------------------------------------------------


------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
------------------------------------------------------------------------