Re: Managed Security Metrics

["Marcus J. Ranum" <mjr () nfr com>]

> I'd want to verify that whoever I was going with a Security MSP shared
> my own philosophy on what is worthy and what is not worthy of reporting
> and logging.

Here's a tale from my "sad but true" folder...

Last year, I went out to do a demo for a security MSP that shall
remain nameless. This was a pretty high-level demo; there were
technical folks there as well as a lot of their business folks. So it
was a Big Deal Demo. I hauled along a bunch of stuff for the demo,
including a laptop (to run our user interface on) a hub, monitor,
and a 1u rackmount IDS running our NFR 4.x NID product. I set
the whole lot up, hooked it to a projector, and started the demo.
3/4 of the way through, someone suggested, "hey, why not plug
it into OUR network HERE?" so I did.

Immediately, or close to immediately, the NID popped up an
alert complaining that a particular MAC address was sending
IP traffic with a broadcast source address. Then it started to
complain that the same MAC address had the IP address of
another system (or at least another MAC address). I was thinking,
"well, -that- ought to show -them-" when the technical guy from
the Security MSP said, "see? those are the kind of false positives
that drive us nuts about IDS products!"
"huh? that's not a false positive! that's an indication that you have
one very screwed up computer and it's possibly screwing up
another right now!"
"well, if we gave that kind of information to our customers, it would
drive them nuts. can you turn that off in your product so that it
wouldn't be so annoying?"
"what kind of information -do- you give your customers, then?"
"we give them a count of the number of alerts that we find based
on searching firewall logs with some in-house developed scripts..."

So - yes - it's extremely important for your Security MSP to share
your idea of what's significant versus insignificant. One thing I think
I'd look for in an SLA would be some kind of feedback loop
whereby the MSP would notify the customer if they decided to
start discarding a particular alert, and wherein they would query
the customer if there was a new alert capability deployed. "do
you want us to notify you about attempts to blah blah your
web server?" (or maybe just a pro-active notification: "we are now
going to notify you of attempts to foo your bar unless you tell
us otherwise.")

mjr.
---
Marcus J. Ranum,  Chief Technology Officer, Network Flight Recorder, Inc.
Work:  http://www.nfr.net
Play: http://www.ranum.com

_______________________________________________
firewall-wizards mailing list
firewall-wizards () nfr com
http://www.nfr.com/mailman/listinfo/firewall-wizards