Logfile retention (was Re: Top-down vs. bottom up (IDS) management)

[Bennett Todd <bet () rahul net>]

1998-04-21-23:20:56 William Stout:
> [ a really neat statement, I resoundingly agree with most of it, but... ]
> An IDS system should also collect only as much data as possible until it
> detects something worth the cost of detailed logging.  We could collect
> all network traffic always, but that would constitute a self-inflicted
> denial-of-storage attack.

This one point I disagree with, fairly strongly. Disk is _Cheap_. Aren't
23GB drives down near $2,000 these days?

Capturing all network traffic may well be impractical, and doing
post-mortem data mining on it is Not Much Fun.

But logfiles really don't grow all that fast, when you have multi-GB
drives lying around. If you can afford the space, collect all
possibly-helpful logs. Rotate daily, compress when you rotate. When the
drive fills up (months? years?) stage old ones to tape. Keep 'em for
months or years. Garnish your logs with useful goodies. My favourite
addition on those lines is Weitse Venema's logdaemon; get all logins
into syslog, and gather 'em all in from every machine in your net and
keep 'em forever.

I'm inspired here by a recent experience. There was a configuration
error on an access router, that had modems hooked up to it. The
configuration error was such that the router _appeared_ to be providing
a nice tight interface; it demanded SecurID authentication and processed
it correctly, only giving you a terminal server prompt after you'd
negotiated that dance successfully. But if you ignored the login prompt
and immediately started trying PPP negotiation, it happily set up the
PPP session with no authentication whatsoever. Ouch.

So naturally we closed the hole instantly. Thanks to cron-driven CVS
logging of the router configurations it was pretty easy to determine the
time window during which this error left a horrifying hole in the
perimeter. And thanks to heavy-handed, paranoid logfile retention we
could look back in time and confirm that nobody else had stumbled across
that hole in the months that it was open.

Disk is cheap.

-Bennett