Re: Snort against DARPA Dataset

[Sunny Fugate <fugate () unm edu>]

Hi Sravan, 

Regarding playing tcpdump files...you can use tcpreplay and simply attach your scan detection tool to an actual (or 
virtual) interface. My default test setup generally uses two NICs (either on the same machine or different machines 
depending on what I want to test), with one sending using tcpreplay and one listening in promiscuous-mode with my 
detector of choice attached. 

However, using TCPreplay can introduce packet drops (and thus false-negatives) that won't be present when reading 
directly from a file.  To limit this tcpreplay has options for controlling packets-per-second and packet-count so you 
can roughly simulate various network loads to minimize packet loss.  

Regarding your detection rates, check that you have signatures for the unidentified traffic.  Is it 30% of labelled 
attacks for which you have signatures, or 30% of the labelled attacks don't have signatures in Snort?   As Robert 
pointed out, many of the old DARPA attacks may not be handled by current detection rules or current preprocessors.  It 
may also be that you might need to change/refine configuration of various specialized pre-processors. Some immediate 
things to check might be port-lists for various preprocessors which might prevent certain preprocessors and/or rules 
from being applied if traffic is not on an expected port.   You'll need to examine your missed attacks, see if these 
are handled at all by Snort and by which preprocessor and whether the preprocessor is configured such that it would 
detect them.  

Cheers,

Sunny

On Jul 2, 2012, at 8:21 AM, Sravan Bhamidipati wrote:

> Thank you, Robert.
>
> Are there any recommended portscan detection tools that can play tcpdump files? I have tried scanlogd and psad, and 
> didn't find the option.
>
> I have a more generic question, which is actually what I'm trying to learn using the DARPA dataset: Given a labeled 
> dataset, what are the ways to tune Snort to achieve the best possible detection rate? Other than turning on all 
> preprocessors, enabling all rules, and configuring frag3 and stream5 bind_to addresses are there any recommendations?
>
> Regards,
> Sravan
>
>
> On Fri, Jun 29, 2012 at 11:49 AM, Robert Vineyard <vineyard () tuffmail com> wrote:
> On 6/29/2012 11:22 AM, Sravan Bhamidipati wrote:
> > 1. Portscan.log: The default Snort logs do not contain sfportscan alerts. Is this by design or can this behavior be 
> > changed? I am using the preprocessor's logfile option for portscan-related attacks. How reliable are the port 
> > ranges and open ports in this log? Do they identify all ports or only a few ports?
> >
> > 2. Detection rates: I am using the 3-tuple (date, source IP, destination IP) as matching criteria for 
> > portscan-related attacks (portscan.log), and the 5-tuple (date, source IP, source port, destination IP, destination 
> > port) as a matching criteria for all other alerts. I see more than 30% of the labeled attacks going unidentified by 
> > Snort. Is this matching criteria correct or in some way too liberal or stringent?
>
> IMHO, port-scan detection is much more easily and efficiently done using netflow analysis tools. I could be wrong, 
> but I'd guess that's why you don't see a lot of feature enhancements to that preprocessor these days.
>
> > 3. Ruleset: How different are the Snort subscriber's ruleset, Pulled Pork rules, and Emerging Threats ruleset? 
> > Would the detection rates improve if I used all rulesets together? (As I understand Snort ignores the older or 
> > duplicate rules.) In general are older signatures (from 1998/99) ever removed or only replaced by newer signatures 
> > in these rulesets?
>
> Pretty different. There will inevitably be some overlap, but Pulled Pork can help you sort things out. It really 
> depends on what you're looking for, so it's hard to say if one is "better" than another. If you're looking for 
> *everything* then you're talking at least 40,000 rules - combining GPL + VRT + ET, and that's not even counting 
> options from other third parties. To make that happen, you're going to need a ton of RAM, and some fairly significant 
> horsepower to chew through that many signatures.
>
> I would say that with a task like that, your first job is to not drop packets. However, since you're replaying canned 
> data, you already have the luxury of a 100% capture rate :-)
>
> > 6. Is it fair to test any IDS against such old datasets?
>
> Are those attacks still seen in the wild? If so, then a modern IDS should be able to detect something from 1998 with 
> no problems.
>
> Just my 2c.
>
> -- Robert Vineyard
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest Snort news!


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!