Re: The State of Information Security, 2004 (survey)

[Rebs Guarina <rebs.guarina () gmail com>]

On Sun, 05 Sep 2004 20:43:16 -0400, Marcus J. Ranum <mjr () ranum com> wrote:
> Speaking of "Bad Surveys" here's a classic. This just came
> across the radar screen this evening....

<snip>

> > Methodology
> >
> > The State of Information Security 2004, a worldwide study by CIO Magazine
> > and PricewaterhouseCoopers, was conducted online from March 22 through
> > April 30, 2004. Readers of CIO Magazine, CSO Magazine and clients of
> > PricewaterhouseCoopers from around the globe were invited via email to take
> > the survey. The results shown in this report are based on the responses of
> > more than 8,000 CEOs, CFOs, CIOs, CSOs, vice presidents and directors of IT
> > and information security from 62 countries. The margin of error for this
> > study is 1%.
>
> OK, this is the important part, here. Can you say "self selected sample"??
> Readers were emailed a survey and some of them answered. What have
> we measured, here?

I actuall agree with your opinion; they have measured the responses
from people who had the time to reply to their email, i.e. those who
did not think that it was spam. Also, 8,000 is a statistically
doubtful number. I've managed 2 national surveys, 3 regional surveys,
dozens of others, and have read tons of statistical papers (being a
statistician, and all); I still have to see a validly published paper
whose survey sample ends in zero. I guess their sampling was biased
from the start. You'll just get biased answers and biased results from
your queries here.....

> One possible thing we have measured is the number of IT executives
> that have spam-blockers. ;)  Or we have measured the number of IT
> executives who have too much free time on their hands... Or - well,
> we don't KNOW - that's the problem with self-selected samples.
> Of particular interest in the description above is the "clients of
> PricewaterhouseCoopers from around the globe" - what does that
> mean? Are they executives, or software engineers or sales reps
> or - what? Again, we don't know. But the premise and tone of the
> survey makes it sound like it's a scientific survey of senior executives;
> reading the "Methodology" makes one wonder if that's the case. My
> guess is even the folks who did the survey have very little actual
> idea what the sampling bias was, here.


-- 
I'm always a newbie. I wouldn't know a damn thing if I were not!
_______________________________________________
firewall-wizards mailing list
firewall-wizards () honor icsalabs com
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards