RE: IPS comparison

["Joseph Hamm" <jhamm () lancope com>]

Sorry for the delayed response.  Comments Inline:
 
> It might, but it might not. All that the anomaly says is that something
has changed. Don't get me wrong, that is useful, but it's not a zero day
exploit. It could be:
>

Administrators are concerned with new threats of all kinds.  The zero
day variety is only a small portion of the things that are detected.

> - a 1000 day old exploit

I'd still want to know about this (IDS should have caught it)

> - someone upgrading the DNS server to also be a WINS server

Think change control management and enforcement

> - some script that got executed which failed over a DNS server to some
>   sort of Tivilo software distribution server

Again, useful to know

> - some sort of switch/hub/network issue which causes UDP broadcast
>   addresses to leak out and 'spike' traffic

I sure as hell want to know about this

> - maybe someone decides to bulk update some zone records in the DNS
>   server with HTTP, SCP or possibly TFTP. Since it doesn't normally
>   do this, this is flagged as an anomaly.
> - maybe one day, the 'new guy' telnets into the box from his house
>   and ends up running ntop on the DNS pinning the session open for a
>   long time and this gets flagged because it's not normal
> - maybe one day, the hard drive crashes, and all the network starts
>   doing DNS requests to the backup DNS server which looks like some
>   sort of DNS worm since you have a large flash crowd going to the
>   same place at the same time.

You just sold the product.  Can I use these examples? LOL!

> Don't get me wrong, I'm a big fan of Lancope's Stealthwatch product and
anomaly detection. We built an anomaly engine into our Thunder log
analysis tool for network
> traffic, netflow, firewall logs, host logs, .etc, but anomaly detection
is just that -- anomalies. It's not the end-all solution to finding zero
days.

No solution is the end all...we all know that!  I think it is funny that
we have to keep mentioning this as if it's a revelation.

> You get legitimate dark IP addresses lighting up all the time.

Then it is no longer dark IP space, is it?  The system should keep track
of this.

> My point here is scale. Unless your NBAD is hooked into your network's
change control process, you won't really know when a new host, network
trace or an entirely new
> route is a legitimate thing or not. It simply wasn't there the day
before and is now.

Yep, these technologies allow administrators to audit change control,
etc.  Great for compliance issues.

> I agree that honeypots are a useful detection concept. I'd be curious
to look at how much fidelity these NDABs (I don't like that acronym any
more than NADS) have at
> seeing if a host is truly alive, especially in this day of DHCP and
host-based firewalls.

It is a passive technology so it is "alive" if you see it communicate
across the network.  IDS/IPS can only detect presence if the traffic
passes the senor.  Leveraging flows from all routers and switches gives
you complete network visibility.  If it makes a peep, you'll see it.

> I've also seen more than one "NDAB" system get really confused if a
system was alive or not because a firewall was silently dropping the TCP
connection attempts.

Sounds like limited visibilty.....a result of poor design and
implementation.

> Agreed, but I most likely already have port scanning detection
algorithms in my firewalls and NIDS.

Yeah, but can you prioritize them? Break down the scanning to determine
the aggressiveness of the scan?  Firewall and IDS alarm data is very
limited in detail.

> With all the network phone-home spyware, custom trojans, custom worms,
constant new forms of P2P chat/share, .etc, on 'unmanaged' networks, I
see so many 'annomalies'
> each day, it's like running a signature NIDS. 

That's why having a NADS to prioritize these anomalies could save you
some man-hours.  There are a lot of anomalies on the network.  You need
to know which ones are of concern.  That's what a tool like this
provides.

> I've yet to see any NBAD vendor call CERT, a product vendor or whoever
and say "Gee, our anomaly engine caught a zero-day worm exploiting
application XYZ. We think you >should do a patch and release an
advisory".
> And it wouldn't even need to be the vendor. Why aren't any of the
customers running these NBAD sensors seeing zero-day exploits and
posting to the SANS GIAC watch, this >IDS list, or even vuln-dev? I
think that NBADs are seeing anomalies just fine, but it's hard to
discriminate an "ignorable" anomaly from a real threat.

It is a quickly emerging space so stay tuned!


Joe Hamm, CISSP
Senior Security Engineer
Lancope, Inc.
jhamm () lancope com
404.644.7227  (cell)
770.225.6509   (fax)

Lancope - Security through Network Intelligence(tm)
StealthWatch(tm) by Lancope, a next-generation network security
solution, delivers behavior-based intrusion detection, policy
enforcement and insightful network analysis.  Visit www.lancope.com.


-----Original Message-----
From: Ron Gula [mailto:rgula () tenablesecurity com] 
Sent: Tuesday, August 30, 2005 9:00 PM
To: Focus-Ids Mailing List
Subject: RE: IPS comparison

OK, here goes a real long post from Ron. Sorry if this gets off topic
but I hope folks find it informative. For anyone just scanning these
emails, let me summarize by saying: "anomaly detection" != "zero day
detection".

At 05:15 PM 8/30/2005, Joseph Hamm wrote:
> > - I agree that "anomaly detection" != "zero day" detection. Just
> because
> >   my DNS server starts to connect to all the other hosts on my
> network,
> >   doesn't mean it has got a worm on it.
>
> It might if your DNS server doesn't normally do this.

It might, but it might not. All that the anomaly says is that something
has changed. Don't get me wrong, that is useful, but it's not a zero day
exploit. It could be:

- a 1000 day old exploit
- someone upgrading the DNS server to also be a WINS server
- some script that got executed which failed over a DNS server to some
   sort of Tivilo software distribution server
- some sort of switch/hub/network issue which causes UDP broadcast
   addresses to leak out and 'spike' traffic
- maybe someone decides to bulk update some zone records in the DNS
   server with HTTP, SCP or possibly TFTP. Since it doesn't normally
   do this, this is flagged as an anomaly.
- maybe one day, the 'new guy' telnets into the box from his house
   and ends up running ntop on the DNS pinning the session open for a
   long time and this gets flagged because it's not normal
- maybe one day, the hard drive crashes, and all the network starts
   doing DNS requests to the backup DNS server which looks like some
   sort of DNS worm since you have a large flash crowd going to the
   same place at the same time.

Don't get me wrong, I'm a big fan of Lancope's Stealthwatch product and
anomaly detection. We built an anomaly engine into our Thunder log
analysis tool for network traffic, netflow, firewall logs, host logs,
.etc, but anomaly detection is just that -- anomalies. It's not the
end-all solution to finding zero days.

> Consider a
> network anomaly detection system that profiled your network and created

> unique usage/behavioral profiles for each host (including Ron's DNS 
> server).  You've told the box how your internal address space is 
> defined so now the box also knows your network's "dark" or unused
address space.
> (Defined internal address space) - (active hosts seen) = Unused or 
> "Dark" IP space

Tenable has got a lot of experience in this space with both active and
passive solutions. Well managed networks are extremely well behaved and
any anomaly is an interesting thing. Unmanaged networks are chaotic,
random and unpredictable. You get legitimate dark IP addresses lighting
up all the time.

My point here is scale. Unless your NBAD is hooked into your network's
change control process, you won't really know when a new host, network
trace or an entirely new route is a legitimate thing or not. It simply
wasn't there the day before and is now.

> Now consider that your DNS server gets infected with a new worm and 
> starts scanning nearby hosts.  A smart NADS (network anomaly detection
> system) will be able to quickly determine that this host is scanning 
> your dark address space (honeynet concept).  Any activity being 
> directed at this unused space can be assumed to be suspect.

This is useful, but a lot of this functionality is already in a NIDS.
They may not profile my network, but they can tell me if my DNS server
is launching port scans.

I agree that honeypots are a useful detection concept. I'd be curious to
look at how much fidelity these NDABs (I don't like that acronym any
more than NADS) have at seeing if a host is truly alive, especially in
this day of DHCP and host-based firewalls.

I have a lot of "by definition" honeypot horror stories though. For
example, consider Windows DHCP laptops that hop from wireless NIC, to
VPN, to ethernet, .etc. Lots of these guys will make various UDP
protocol probes for whatever private RFC1918 address they were last on,
or misconfigured for. I've also seen more than one "NDAB" system get
really confused if a system was alive or not because a firewall was
silently dropping the TCP connection attempts.

> Also, consider the port being used.  If this is not a port that the DNS

> server normally uses, then that would be suspect as well.

I normally don't need to download to patch my DNS server, but right
around the time a worm for DNS is going to come around, I'll have to
fire up my infrequently used VNC, SSH, terminal services, Telnet or
rlogin. I might even have to grab a patch with a web fetch on port 80,
port 443, or even go through the new corporate web proxy on some other
god-forsaken-port.
I may even have a system admin who's arguably lazy or efficient and
batch process this with a TFTP fetch.

The classic example I see a lot of NBAD, IDS and SIM vendors (I've seen
more than one Tenable SE guilty of this too) is to demonstrate to
potential customers when a perfectly 'normal' host gets an 'anomaly' of
a web fetch to the network or an IRC session or some TFTP requests. This
is cool, but something easily blocked by a firewall or router policy.

> Not to
> mention all of the tcp resets or icmp port unreachables (in the case of
> UDP) that the DNS server will receive as it scans across your network.

Agreed, but I most likely already have port scanning detection
algorithms in my firewalls and NIDS.

> Check out the whitepapers published by NADS vendors that outline cases 
> where new "zero-day" worms were detected before signatures became 
> available.  I agree that marketing departments squeeze all they can out

> of zero-day, but I can at least vouch that our SteatlhWatch products 
> detected Zotob (and its variants) without the need for any signature 
> updates.  This means customers had early detection before signatures 
> were available.  I'm sure there are some other vendors out there that 
> can report the same.

Many NIDS, SIMs, .etc (even Tenable's NeVO) caught various hits and
indications of Zotob. What they didn't do is say, "this trace right here
is the one that is taking your network down right now and will cause you
loss of sleep for the next three days".

With all the network phone-home spyware, custom trojans, custom worms,
constant new forms of P2P chat/share, .etc, on 'unmanaged' networks, I
see so many 'annomalies' each day, it's like running a signature NIDS.
On a well managed, firewalled, change-controlled server farm, it's a
different story.

> Even signature based solutions can tout this if they write sigs for the

> vulnerability as opposed to the exploit.

Sure. Anyone can tout just about anything in this industry. However,
some things remain to be seen in this industry.

I've yet to see any NBAD vendor call CERT, a product vendor or whoever
and say "Gee, our anomaly engine caught a zero-day worm exploiting
application XYZ. We think you should do a patch and release an
advisory".
And it wouldn't even need to be the vendor. Why aren't any of the
customers running these NBAD sensors seeing zero-day exploits and
posting to the SANS GIAC watch, this IDS list, or even vuln-dev? I think
that NBADs are seeing anomalies just fine, but it's hard to discriminate
an "ignorable" anomaly from a real threat.

Ron Gula, CTO
Tenable Network Security


------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
------------------------------------------------------------------------


------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
------------------------------------------------------------------------