Re: automatic signature generation

["Sanjay R" <2sanjayr () gmail com>]

On 5/22/07, Eric Hacker <focus () erichacker com> wrote:
> On 5/20/07, Sanjay R <2sanjayr () gmail com> wrote:
> > Hi List:
> > There have been few studies to propose the automatic generation for
> > misuse based IDS, like snort (in fact, it is the hot area of research
> > among IDS researchers). Suddenly, it came into my mind, whether is it
> > feasible to generate (Good) signatures for all types of attack in an
> > automatic way (in a black-box environment, where we don't have the
> > source-code of the vulnerable application)?
>
> Sanjay, please define feasible.
to me feasible means creating signatures in timely manner so that they
can be deployed soon to control the attacks.
> Let's look just at network IDS. There is no doubt that one can take a
> packet or session and create a signature to match. The question is
> will it match other packets exploiting that vulnerability or match
> other packets not. Thus what is a feasible rate of false positives
> (F+) and false negatives (F-). The problem is that depending on
> circumstances, acceptable F+ and F- are variable.
>
> For instance, immediately after a worm exploiting an unpatchable zero
> day vulnerability is released, one might have a fairly high tolerance
> for false positives in detection mode and find false negatives
> unacceptable. "Better safe than sorry" so to speak. After the risk has
> died down somewhat, then F+ becomes annoying and hampers the
> effectiveness of the analysts. F- might be unacceptable with a high
> risk, and more tolerable when the risk goes down, especially if a
> victim makes a whole lot of noise anyway.
>
> > The main cause of my doubt is the observation that it is not
> > feasible to generate attacks automatically. Usually, an attacker spend
> > hours to analyze the application and then write an exploit.
>
> Just because you haven't seen it done doesn't mean it can't be done
> (or isn't being done). Look at the evolution of reverse engineering
> patches and creating exploits in recent years. Automated generation of
> exploits is a matter of resource allocation. If this was a wartime
> need, it would be done already. It will happen; we should be thankful
> it hasn't yet.
its true and thats why i used the term "feasible." what i wanted to
mention is that writing a exploit is dependent on the application and
the vulnerability. it is not common for all applications. therefore,
there has to be a very complex logic to implement an automated way of
generating exploits. Similarly (my implication) if we want our
hypothetical IDS to generate (and update) its signatures
automatically, we may require a very complex logic. so, if it is
feasible, then what will be the components of such an IDS? this is
what I am planning to work on.
> > We don't
> > have any tool that take, as an input,  the application to be
> > exploited, and gives us an working exploit (of course, Metasploit
> > helps us to create exploit). Therefore, the early thought that comes
> > into my mind is "creating an automated signature generation tool is as
> > difficult as creating an automated attack generation tool". I would
> > like to know your opinion on this.
>
> The main problem I see is that we don't have widespread tools to
> automate the test/QA process for verifying that the generated
> signature is feasible. Regardless of what your definition of feasible
> is, if testing the signature is manual (or only live production) then
> one can hardly say that the process is automated.
agreed. I would like to use the term "signature evolution." its true
that we can not test the signature immediately, but we can allow it to
evolve to become a "good" signature (i.e. tolerate some FPs, as u
mentioned earlier). remember, that as signatures are generated
automatically, any FN will either trigger another new
signature_generation or update the existing signature (i have some
concerns here though)
> Where is the "make test" for IDS signatures?
>
> There are necessary pieces, but I don't think that there is anything
> available to tie them together except unique shell scripts for each
> one. There is tcpreplay for playing captures, but openpacket.org is
> still waiting to be born. One can attach files to signatures at
> doc.bleedingthreats.net but that doesn't help much for normal network
> traffic.
>
> Sometimes a packet replay isn't good enough in any case. One really
> needs real time coordinated activities between the attacker, victim
> and IDS is order to support automated testing of a signature. The
> process is:
>  Launch attack
>  Get IDS response.
>  Get victim effect.
>  Does response match effect?
>  Lather, rinse, repeat.....
its true. And I think that automated IDS should know the health of the
victim. therefore, in a way, it should be a HIDS +NIDS.
> Packet captures can sometimes represent an known victim effect, but
> sometimes they can't. And unless our friendly list advertiser (Core)
> has updated their testing capabilities with real time log coordination
> recently, then I don't think a "feasible" automated IDS testing tool
> is available commercially or open.
>
> However, I am getting close to making one available. I've got
> proposals in the works for end of summer security conferences. Even if
> my system for IDS signature QA doesn't really meet the need, it will
> be open source and easy for others to learn from my mistakes. And I'm
> probably wrong about there being other tools too; I'm not perfect.
>
> So, Sanjay, if by the end of the summer you have implemented a cool
> algorithm for automated IDS signature generation, I'll set you up with
> a "make test" script that will automatically coordinate the attacker,
> IDS and victim to see if it passes.
great!! but summer is too early for me. I have just started thinking
about this project.
> The only thing really in the way is not believing it can be done.
well.... :)
thanks Eric
-Sanjay

> Peace,
> --
> Eric Hacker, CISSP
>
> aptronym (AP-troh-NIM) noun
> A name that is especially suited to the profession of its owner
>
> I _can_ leave well enough alone, but my criteria for well enough is
> pretty darn high.


--
Postdoc, DIT, University of Trento, Italy

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw 
to learn more.
------------------------------------------------------------------------