Re: Managed Security Metrics

["Jack McCarthy" <Security.Info () jackmccarthy com>]

Hey Mike,

     Here are some of my thoughts when I read your post.  Some may be helpful and some are just that, thoughts.


This is a link to a product called WebTrends, it provides Firewall Security Report - now I'm not promoting their 
product but just using the report that product produces as a guideline to some of the metrics you may be looking for.
<http://www.webtrends.com/reports/reports.asp?reports=product&product=fire>


If you haven’t done so already, you might want to throw this topic to a few other lists, like:

-Security How-to List
-E-Commerce Security List
Can be found at
<http://www.ntsecurity.net/go/loader.asp?id=/security/subscribe-ntsd1.htm>.

SecurityFocus.com has some good mailing lists, but you probably know about these too.


(now standing in your shoes) As one of the  ‘customers’ of this outsourcing co. I want or would like:

-To know all statistics – everything from pings to scans to attempts.  Anything that touches my border, I want to know 
about. Maybe in a weekly or monthly report, with exception of course for attacks and intrusions where immediate 
notification would be justified.  <This may be asking too much, but a good place to begin if you have to start 
backing-off of a few things.>

-Accountability – checks and balances. I want to be able to check and make sure you’re doing your job and doing it 
correctly. Some way of independently checking on you, your reports and findings.  <I think self education on the 
material/technology and real-time access just like they have is the most accurate ‘checks and balances’.>  A way to be 
able to check if the latest patches, service packs, upgrades are installed or exploits plugged.  I want to be able to 
verify it myself, not just take your word for it.  <A certain level of paranoia is a good thing when it comes to 
security.>

Some things that concern me, again being the ‘customer’ and in your shoes:

-Relying 100% on a vender to do business in my best interest and me not knowing enough of the material/technology to 
know that they’re feeding me BS.  There needs to be some way of checking up on them.

Questions/comments I might ask/say or just think about:

-How do I know your not fudging your reports/finding to support my interests?  I don’t mean to be so blunt, but we’re 
not here to play bingo.

-May I talk to some of your current clients?  Can I have a list of current clients that I can talk to.

-Can I get something on-site that I can view the logs and reports in real-time?

-How do you guys keep up with the latest exploits, patches, service packs and upgrades?

-How quickly after an exploit has been made public (or known about) do you patch the hole?

-What happens if there is a breach of security, WHICH THERE WON’T BE but..?  What do you do?  How do you handle it?

-Do you have any experience in tracking down attackers and intruders?

-How successful have you been in finding the source of the attack or locating the intruder?


Again, these are just some of the thoughts that ran through my head after reading your post but, having these questions 
answered (with the answers you want) on paper and signed by them would be one of my highest priorities.  Hope this 
helps.

-Jack McCarthy


--- Mike Smith <msmith () infinity-its com> wrote:
> What security metrics should I be looking for in a service level agreement
> from a managed security service provider?  Traditional service level
> agreements cover things like performance (throughput) and availability.  If
> I have an outsourcer manage my firewall, what kinds of service targets
> should I insist on?
>
> I wouldn't think there'd be any point to counting blocked attacks (as a
> service metric).  I certainly want to know how many attacks got through, but
> is that a metric for which I can usefully set a target (e.g., no more than 0
> successful attacks per month)?
>
> If the service provider manages my firewall plus other devices, like VPNs,
> IDSes, etc., can  we or should we set different types of targets for each
> device/service?  Or should there be some global security metrics that apply
> across the entire service?  I'd like to know how much of my bandwidth I'm
> giving up to the security provider's data streams, but that doesn't tell me
> how secure I am.
>
> Related to this, I recently listened to a Meta audio briefing
> (http://www.metagroup.com/metaview/mv0314/mv0314.html) that suggested some
> useful security metrics (aimed more at internal security operations)
> included things like password reset requests, time to create or delete user
> accounts, etc.  Would these work for measuring an external service provider?
>
> J. Michael Smith



_______________________________________________
firewall-wizards mailing list
firewall-wizards () nfr com
http://www.nfr.com/mailman/listinfo/firewall-wizards