RE: Log checking?

["Paul D. Robertson" <paul () compuwar net>]

On Fri, 1 Oct 2004, Marcus J. Ranum wrote:

> Ranum's first law of Log Analysis:
>         - Never keep more than you can conceive of possibly looking at

Never keep, or never capture?  All the time, or due to a point incident?
If you had all the resources you want, or with the resources you have?

> Ranum's second law of Log Analysis:
>         - The number of times an uninteresting thing happens is an interesting
>                 thing

Robertson's corollary:

Lots of things can be interesting, they aren't all useful, and those that
are aren't always useful enough to spend time on.  Better to spend time on
the most interesting rather than any interesting things.

> Ranum's third law of Log Analysis:
>         - Keep everything you possibly can except for where you come
>                 into conflict with the First Law

Robertson's corollary:

Keeping things has a cost and a risk, assess that before you decide
keeping everything is a good idea.  Sunsets are often pretty, even for
data retention[1].

> [#insert plug for my log analysis tutorial at USENIX and SANS
> see http://www.loganalysis.org/news/tutorials for details]
>
> > While I generally recommend folks log as much as possible, with specific
> > sunsets on retention, if you have 5,000 script kiddie attacks a day, you
> > tend to evaluate where and what logging is important in a different light.
>
> The number of times an uninteresting thing happens is an interesting
> thing. The number 5,000 in your example above is an interesting
> number and you wouldn't have it available to you if you hadn't
> counted it. It might, for example, be interesting if it went to 10,000.
> It might be even MORE interesting if it went to 0. ;)

It might break the logging infrastructure if it went to 10,000 too- but
for operational networks, I'm more concerned with keeping successful
attacks at zero than I am curious about the level of unsuccessful attacks.

Sampling is often cheaper than monitoring, and while it adds skew, it may
be a more productive use of resources, especially if monitoring requires
massively expensive infrastructure upgrades[2].

> > Now, if you're not sued often, the idea of discoverable information may
> > not be all that much of an issue- but if you've dealt with fulfilling
> > discovery motions, you'll not want to have to excerpt terabytes of logs
> > for every fishing expedition a lawyer might mount.
>
> 1) Judges are getting a log better about not allowing massive
>         fishing expeditions

Some are, some aren't.  We're making some progress, but in a lot of cases,
the onus is on You to find and report the relevant information, not a
third party, or the other side, so while you can get away with "here it
all is" in some cases, in others the discovery order's narrowness compels
you to produce the relevant data, and that costs resources.

> 2) Who cares if someone wants to discover what you rightly
>         describe as "script kiddie" activity? Give 'em a terabyte
>         and let them have fun with it!

You can't always get away with that- and indeed depending on the court's
order that may be contempt.  So, do I really want to spend my resources
deciding what's in scope for a subpoena sifting through large amounts of
data[3]?  Sometimes it's better to be able to say "Sorry, don't have any
of that!"

> The problem is that you're not analyzing the problem methodically.

> From your perspective, from my perspective, I've analyzed the problem and
decided that when I'm running an operational network, the number of
interesting things is different than when I'm running a research network,
because my organization's priorities and capability to do work is
different (see your own first law.)

> If you care about that kind of stuff, just keep internal logs differently
> from external, etc. You might just keep counts of one type of
> data, versus actual data in another case - and you need to make
> these decisions rationally based on your site's security
> needs, bandwidth usage, event load, and legal concerns - not
> just because someone on Firewall-Wizards said to or not to. ;)

Yep, which is why I outlined both the good and bad points of doing so-
but mostly the whole point of bringing it up is that people
tend to focus on blocked logs when looking at passed stuff may be
considerably more interesting- and a bigger threat- NFR was way too far
ahead of the market, that capability is starting to become more
interesting to more people now. :-P  My last post contained a fair number
of reasons why you might keep and look at different things, along with why
I personally choose not to.

> > Only you can make a determination for your organization if say logging
> > automated probes where there are no accessible systems is "worth it."
>
> Right!

I'd suspect that if you actually looked at utilization and cost, the
answer for lots of organizations would be "nope!"

Should the cost of a control have any relation to the cost of not having
that control?  How do you quantify the value of interesting?

On my last operational watch, I spent years silently dropping stuff at the
border that you'd call interesting, and I never had a compromise via a
mechanism I was able to stop (viruses weren't in my charter- the company
couldn't deal with my blocking policies, so I simply transited mail)-
while similar organizations with almost identical profiles and
expenditures were compromised.  We were all targeted by the same attackers
as far as I could tell (though I may have had a few extra labor-sponsored
attempts than many of my peers due to a particularly long and nasty
strike.)  So, you're going to have a hard time convincing me that I needed
to spend >$200,000 analyzing things that I didn't analyze when the folks
that spent their time on that got bitten because it took resources from
doing other things that I found more interesting ;).

Paul
[1] Living through the subpoena-receiving side of Iran-Contra gave me
early insight into this.
[2] At the height of the directed attack period of my last employer, we'd
have spent about the equivalent of the cost of the Internet infrastructure
and then some to be able to monitor everything, and that was almost all
pointed at a then unprofitable business.  I recall a certain product vendor
*cough* *cough* not being interested enough to try to help fix that problem
with the only product that had a chance of doing so at that time.
[3] See [1].
-----------------------------------------------------------------------------
Paul D. Robertson      "My statements in this message are personal opinions
paul () compuwar net       which may have no basis whatsoever in fact."
probertson () trusecure com Director of Risk Assessment TruSecure Corporation
_______________________________________________
firewall-wizards mailing list
firewall-wizards () honor icsalabs com
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards