Re: MIT Darpa Dataset,

[Sanjay Rawat <sanjayr () intoto com>]

At 02:39 PM 9/19/2005, Wilmar SULAIMAN wrote:
> Dear all,
>
> I was wondering how normally people test their machine learning 
> algorithims in MIT darpa dataset?

A lot of preprocessing is required before you actually apply DARPA data to 
any of the ML algos. In my case, it took a major portion of total time, I 
spent in experimentation!!!


> I asked this question because I found that :
> 1) There is a time gap between the published list of attack to the 
> dataset. For instance APACHE2 attack, the published list of attack say the 
> attack was start on 00:00:40 (not the fake time), normally I see the 
> attack within +-10 seconds of the published list of attack (this is by 
> looking directly on the payload)
>
> 2) Many of U2R attack are in port 23. For instance the attacker executes 
> the command : $./exploit
> In the real dataset especially for per packet model, it will get 
> translated to many packets i.e
> 1st packet contains "$"
> 2st packet contains "."
> 3rd packet contains "/"
> 4th packet contains "e"
>
> If the IDS identified say the 4th packet as anomalous , is it justifiable 
> to say that the IDS detects the particular attack?

It depends on the implementation of your IDS. Some IDS buffer the data 
before matching it against the stored patterns. In this case, one should be 
able to see the whole string "$./exploit". So..no problem. If you are 
looking per packet basis and you are able to find some anomalous pattern in 
one packet, before the connection is terminated, then I think, it should be 
OK. If your aim is to find intrusive "connections", then any packet 
corresponding to intrusive connection, detected as malicious, should be good.


> thanks
>
> Wilmar Sulaiman
>
>
>
> ------------------------------------------------------------------------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from 
> CORE IMPACT.
> Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
> to learn more.
> ------------------------------------------------------------------------

Sanjay Rawat
Senior Software Engineer
INTOTO Software (India) Private Limited
Uma Plaza, Above HSBC Bank, Nagarjuna Hills
PunjaGutta,Hyderabad 500082 | India
Office: + 91 40 23358927/28 Extn 422
Website : www.intoto.com
  Homepage: http://sanjay-rawat.tripod.com






------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
------------------------------------------------------------------------