RE: Managed Security Metrics

["R. DuFresne" <dufresne () sysinfo com>]

On Mon, 5 Mar 2001, Mike Smith wrote:

> I'm still not sure this is useful to the client of a managed security
> service provider.  If the service level agreement says, "provider shall
> block X attacks per month," what does it mean to me if it blocks less or
> more than X attacks in a given month?  A month could have fewer than X
> blocked attacks because it was a quiet month for attacks in general.  Does
> the provider owe me a rebate for failing to meet the target?  Is there an
> incentive for the provider to surreptitiously encourage/launch blockable
> attacks to boost its performance rating?
>
> For the flip side, I'm hesitant to look for a service guarantee like
> "provider shall block x% of attacks per month" or "provider shall permit no
> more than X attacks to penetrate the firewall per month" because any
> successful attack is unacceptable.  I can't see management signing an
> agreement that says it's okay if some small number of attacks gets through
> the firewall.
>
> Anyway, I think such a requirement would be akin to proving a negative.  How
> can I or the service provider be sure that an attack didn't get through;
> perhaps neither of us has detected it yet.
>
> So I'm back to asking, what are suitable, measurable criteria for judging
> the quality of my security service provider's performance?

I'm sorry if I implied this.  I did not mean to do so, let me try to
clarify.  

With an attempt to give the folks you are *managing* devices for, rather
then to give your *management* some stuff to make neat little charts out
of, how about this;

A comparision of what the outside IDS sensor saw and interprested,
compared to what the 'properly placed' internal IDS system didn't see.
This might prove to be more useful data, then again, isn't it all
marketing?

Thanks,

Ron DuFresne
-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        admin & senior consultant:  darkstar.sysinfo.com
                  http://darkstar.sysinfo.com

"Cutting the space budget really restores my faith in humanity.  It
eliminates dreams, goals, and ideals and lets us get straight to the
business of hate, debauchery, and self-annihilation."
                -- Johnny Hart

testing, only testing, and damn good at it too!

_______________________________________________
firewall-wizards mailing list
firewall-wizards () nfr com
http://www.nfr.com/mailman/listinfo/firewall-wizards