Darkreading: Secure Coding Certification

[Greg.Beeley at LightSys.org (Greg Beeley)]

> I agree that multiple choice alone is inadequate to test the true
> breadth and depth of someone's security knowledge. Having contributed
> a few questions to the SANS pool, I take issue with Gary's article
> when it implies that you can pass the GSSP test while clueless.
>
> There is indeed a body of knowledge that is being tested. SANS has
> been soliciting comments on the document.

Having taught this type of material before at the university and
vocational levels, I think there are three main aspects which are
important to someone's capability to code "securely":

1 - Knowledge of pitfalls, countermeasures, and good practices;
2 - The right mindset; and
3 - Experience carrying it out

(there are also the surrounding business issues, like project management
and planning, risk assessment and how well-vetted the software must be
given the cost/risk scenario, but I'll just stick to the coder for now).

I have not reviewed the GSSP (practice exams, etc.), but I am guessing
that it goes after the "low hanging fruit" of covering (1) above, which
is testable most easily with an exam.  It's much better than nothing, and
the knowledge is very important, but this test does not necessarily mean
that a particular coder will be a better "secure coder".  There's a lot
more to this than just a body of knowledge.

For example, you could give any auto mechanic a test that they could
pass if they know what the risks are in leaving a bolt loose, or a
fuel system clamp unsecured, or not replacing an O-ring when a
connection is open (or, if they can figure out those risks during the
exam, esp. a multiple-choice one).  But that does not mean that the
mechanic will actually follow through with those things, or that, in
practice, the mechanic will actually be more prone to even notice...

So, although I think the GSSP is an important first step, I tend to
agree with Gary.  In my university-level teaching of software
security, I would never even begin to consider evaluating my students
merely via multiple choice exams.  Not with this subject matter.

Greg.