Education and security -- another perspective (was "ACM Queue - Content")

["Wall, Kevin" <Kevin.Wall () qwest com>]

Kenneth R. van Wyk wrote...

> FYI, there's an ACM Queue issue out that focuses on security -- see 
> http://acmqueue.com/modules.php?name=Content&pa=list_pages_issues&issue_id=14
>
> Two articles there that should be of interest to SC-L readers include
> Marcus Ranum's "Security: The root of the problem -- Why is it we can't
> seem to produce secure, high quality code?"  ...<snip>...

I've been thinking alot about some of the statements that Marcus Ranum
made in his most recent article in the _ACM Queue_ (Vol 2, No 4)...
even before Ken invited us all to comment on it.

I mostly agree with Ranum's conclusions, although perhaps for
different reasons.

Ranum states:
        "It's clear to me that we're:
         + Trying to teach programmers how to write more secure code
         + Failing miserably at the task"

He goes on to say that "it [educational approach] flat out hasn't
worked".

In general, I don't think this is an issue that is unique to _secure_
programming (coding, design, etc.). I think over the past 40 years or
so, as a discipline, we've failed rather miserably at teaching
programming, period. For the past 25 years, I've worked closely with
both highly educated Ph.D. computer scientists and with those whose
formal CS education consisted of at most a course or two in something
like C or Pascal. In many of these cases, the less educated are
beating out those who have had more formal education. (In fact,
I'd say this has been true in at least as many as 50% of the cases.)

What makes the difference? Well, it goes beyond mere aptitude and
general intelligence. I think in part at least, it goes with having
a passion for what you do. To some, doing design and coding and
other creative aspects is an artistic expression, a noble cause
and they would do it even if there weren't paid for its--witness
the open source movement which is largely funded by volunteer
labor. Others see it as a "job" or a "career path", but not much
more. In my 25 year observation, those with this PASSION almost
always "get it", and those without it are usually left behind
after the first few years into the profession.

I think that the same can be said for "secure coding / design".
Not only do those people have a passion for coding / design, but
the ones who seem to "get it" are the ones who have a passion for
security as well.

Okay, so probably no surprise here, right? Do what you enjoy and
you'll excel at it more often than ones who do it out of other
motives (no matter how noble--such as making an affordable living to
provide for your family).

So I agree with Ranum in a sense--that educational approaches to
security have overall failed, but I think it is not because the
educational process / system per se has failed us (not that I'm
arguing that it couldn't be improved), but because we haven't been
able to ignite the passion for security in others. (And frankly,
I'm not even to what degree that's possible. I'll leave that to
another discussion.)

In the past two years, I've had the fortune to teach a computer
security course that I had the major part in organizing / developing.
I have learned two things about the students during that time:
        1) All the students do well when it comes to rote
           memorization. (E.g., questions such as "What cipher mode
           doesn't require an Initialization Vector?", etc.)
        2) Only the students that seem to "get it" seem to do well
           on the questions requiring thought (i.e., ones requiring
           reasoning "outside the box").

Surprisingly (at least at first), I have often been discovered that
those who other faculty members often consider the brightest students
are ones who do the worst on the "questions requiring thought".

But in general, by the end of the 12 week period, I usually can tell
who is going to take and try to apply what they learned and those
who just chalk up the course as another 3 credit hours.

I see what I think is a related phenomena in the commercial world
as well. I've worked with a lot of developers who have worked on
security-related software (e.g., firewalls, crypto, secure proxies,
authentication and access control systems, etc.). One would EXPECT
that the groups that work on these projects would as a whole do
better at developing secure programs than the IT industry as a whole.
But overall, I don't think that their batting average is all that
much higher than the industry at large. We often hear excuses for
this ("security software is more complex", etc.), but I'm not buying
it. If anything, it's this observation more than anything else that
makes me think that formal education is not THE answer (although,
I do think it is part of the answer).

On a related note to security and education, I was wondering if anyone
knows of any experimental data that shows that those with formal
education in security develop more secure programs than those
who have never had such formal training?  If no such experimental
data exists, why not? Can no one think of some formal, repeatable
simple experiments that one could do over several years to see
if formal security education has a significant effect on developing
secure programs? Surely there must be something that educators
could do to formally test this. (Similarly with experiments measuring
security of open source vs proprietary source.) Science does make
very much forward progress until then can propose hypothesis and then
test those hypothesis with some real experimental data. IMO, that's
one of the big failings of computer science...very little experimental
data to back up our often unfounded hypothesis...thus we find
ourselves bouncing from one fad to another because as a profession
have no grounding.

Anyway, I've rambled far too long.
-kevin
---
Kevin W. Wall           Qwest Information Technology, Inc.
[EMAIL PROTECTED]       Phone: 614.215.4788
"The reason you have people breaking into your software all 
over the place is because your software sucks..."
 -- Former whitehouse cybersecurity advisor, Richard Clarke,
    at eWeek Security Summit