Re: Measuring security

[Joel Rosenblatt <joel () COLUMBIA EDU>]

Hi,

There is a sub committee of effective practices working on security metrics (I am chairing it.)  We have broken them 
(the metrics) into 4 types

Compliance Metrics
Executive Metrics
Incident Metrics
Operational Metrics

In each, we are looking for important (as defined by us :-) metrics that meet our criteria of

Automate-ability
Reproduce-ability
Non-Subjectivism

We are also looking to develop metrics that can be compared in a meaningful way between various institutions, the 
significance of this is to allow real
comparisons to be done of various security implementations or products.

We will be building a cook book from these metrics (at least 3 in each category) so that by following our recipes, any 
institute (large or small, rich or poor)
could build a meaningful set of metrics and know how they are doing, in a similar way that a doctor can tell something 
about your health by looking at your
blood pressure and various other tests.

I realize that this is a high level (meaning not really useful :-) look at what we are doing, but we should have some 
useful content available soon.

I hope this helps.

Thank you,
Joel Rosenblatt

Joel Rosenblatt, Manager Network & Computer Security
Columbia Information Security Office (CISO)
Columbia University, 612 W 115th Street, NY, NY 10025 / 212 854 3033
http://www.columbia.edu/~joel


--On Wednesday, November 05, 2008 2:05 PM -0800 Heather Flanagan <heatherf () STANFORD EDU> wrote:

> Hi all -
>
> I've been asked to create some measurable target goals for data security.  This is proving to be a tricky set of metrics to 
> define!  What I've realized so
> far is:
>
> 1 - trying to go by how many holes or warnings are found by nessus won't work; way to many false positives
> 2 - trying to go by what a third-party penetration test might find won't work; what they are measuring varies too much 
> and there have so far been way too
> many false positives or things we considered completely acceptable (yes, a domain controller is going to act as a time 
> server to anyone who checks)
> 3 - trying to go by "well, doesn't look like we've been hacked recently"...  not quite the business metric I'm looking 
> for
>
> Is anyone out there finding any particular set of metrics working for you and your campus leadership?
>
> Heather Flanagan
> Director, System Administration
> heatherf () stanford edu



Joel Rosenblatt, Manager Network & Computer Security
Columbia Information Security Office (CISO)
Columbia University, 612 W 115th Street, NY, NY 10025 / 212 854 3033
http://www.columbia.edu/~joel