Re: Graphing: Don't believe everything you see.

[Adam Shostack <adam () homeport org>]

Avery,

I'll know it when I see it. :)

I was really excited to see "Is There a Cost to Privacy Breachs? An
Event Study," Alessandro Acquisti, Allan Friedman, and Rahul
Telang. WEIS 2006 and ICIS 2006.
(http://www.heinz.cmu.edu/~acquisti/papers/acquisti-friedman-telang-privacy-breaches.pdf)
 This study debunked the idea that breach notices hurt the company's
shareholders in the long run.  It's an important mis-conception, and
I'm glad to have data to show that it's wrong.

Similarly, I was pleased to see my co-blogger Chris Walsh refute a
claim about 'the industry's dumbest practice' by looking at data.
(http://www.emergentchaos.com/archives/2006/12/lets_look_at_some_data.html)

So I don't know what I want to see in detail.  But what I want to see,
in a broad sense, is that we get over our shame over having made
mistakes, and start discussing what really goes wrong.  I want to see
us discussing it in a data driven fashion.  Data is not the plural
of anecdote.  Data comes from having a consistent sampling method.
"Compelled by law to disclose, and unable to find a loophole" is
admittedly not the best sampling method, but it's better than
anecdote, and it's better than voluntary anonymous survey.  I hope
that by understanding that the sky isn't falling, we can evolve better
sampling and disclosure, and start to make real progress by studying
problems.

I'll get off my soapbox before Dave kills me now.

Adam


On Wed, Feb 07, 2007 at 09:15:14PM -0500, Avery Sawaba wrote:
| I'm actually doing some analysis on this data right now (I'm
| sawaba () attrition org). Is there anything in particular you'd like to see?
| Perhaps I already have some of what you're looking for, but I haven't posted
| any of my metrics. I can drop a note to the list if/when something is posted.
| 
| --Sawaba
| 
| On 2/7/07, Adam Shostack <adam () homeport org> wrote:
| 
|     Speaking for myself, I think there are much more interesting questions
|     than looking at correlations between defects and complexity.  For
|     example, we could look at correlations between failures in the real
|     world and training/education.
| 
|     The breach notices that Attrition is accumulating
|     (http://attrition.org/dataloss) give us a set of real wolrd failure
|     data.  That's something we've never really had.  Now we can start
|     mining it and learning things.  For example, does the number of CISSPs
|     employed by an organization correlate with the reports of failures
|     compared to other similar orgs?  Is that correlation positive or
|     negative?  Does "user education" have an effect?
| 
|     There's a huge amount of data in the attrition data set, and it all
|     involves real pain that real organizations are feeling as they try to
|     secure their data.  It's worth studying.
| 
|     Adam
| 
|     On Wed, Feb 07, 2007 at 02:35:38AM -0500, dan () geer org wrote:
|     |
|     | If anyone wants to argue about whether complexity
|     | and security are negatively correlated, then let's
|     | get to it.
|     |
|     | --dan, resisting burning bandwidth unasked
|     |
|     | _______________________________________________
|     | Dailydave mailing list
|     | Dailydave () lists immunitysec com
|     | http://lists.immunitysec.com/mailman/listinfo/dailydave
|     _______________________________________________
|     Dailydave mailing list
|     Dailydave () lists immunitysec com
|     http://lists.immunitysec.com/mailman/listinfo/dailydave
| 
| 
_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave