Re: IDS and honeypots

[Christian Kreibich <christian () whoop org>]

Hi,

On Wed, 2003-04-30 at 19:54, Niels Provos wrote:
> On Wed, Apr 30, 2003 at 02:28:17PM +0200, rnoble wrote:
> >    I'm investigating the idea of using the traffic captured by a honeypot
> >    (in theory all data should be suspicious) and filtering out legal
> >    traffic and traffic captured by existing misuse IDS signatures and
> >    using the remainder to automatically create new signatures in order to
> >    update IDS a IDS database
> Look at
>
>   http://niels.xtdnet.nl/honeyd/ch01-results/
>
> Christian Kreibich's Honeycomb does something similar.

hey it seems Niels hit the send button faster than I could :)

I'm currently working on an updated version of the honeyd plugin that I
sumbmitted to the competition. It clearly has to be said that the
creation of these signatures is a hard problem in terms of not producing
too much noise and creating signatures that would instantly be useful in
a production environment -- while at the same time keeping performance
reasonable. Clearly there are many ways you can correlate different
types of network traffic. Also you can either try to optimize for
certain network protocols (say only look at HTTP GETs for HTTP, or
login/password phrases for FTP) or try to come up with a generic
solution, free of protocol-dependant knowledge. The latter is what I'm
currently trying, although a combination would surely be most effective.

What I have working right now basically does the following:

- Keeps a history of previous connections to the honeypot, making a
difference between initialized and established TCP connections (thus a
portscan basically doesn't kick the more interesting connections out of
my hash tables)
- Compares IP, TCP and UDP headers of packets in those previous
connections to spot matching port numbers, IP addresses etc
- Checks header fields for protocol violations [1]
- Applies a longest-common-substring algorithm to packet payloads of
previous connections. I do this both on matching "messages" exchanged
(where a message is all the data going in one way on a TCP connection
without any data going the other way, similarly for UDP) and across TCP
messages sent in one way (this captures similar user/password
combinations for example)
- Correlates detected signatures to minimize the creation of duplicate
or less accurate signatures, i.e. registered signatures can be improved
upon and reported again
- Port number correlation -- the system right now is pretty good at
telling you what destination port numbers where used in similar traffic
patterns -- e.g. for nmap portscans you don't just get signatures
telling you a connection was made to "any" port (as the scan likely
covered more than one port), but the precise ranges.

New releases of the code should be ready in about two weeks; I can post
a note here in case people are interested. I'll also finally get the
chance to actually test the system on real-world traffic in the next few
days.

Regards,
Christian.
[1] see http://www.icir.org/vern/papers/norm-usenix-sec-01-html/
-- 
________________________________________________________________________
                                                    http://www.whoop.org