Value and use of penetration testing and vulnerability assessment in .edu

[Curt Wilson <curtw () SIU EDU>]

Dear Educause security community -

In addition to my work at the university here, I have done work as a
consultant and in that capacity, performed many vulnerability
assessments and some penetration tests. I know that they are sometimes
over-hyped and are not the solution to all of our security issues like
some vendors would have us believe, however I have seen significant
value delivered through these practices, especially when an organization
does not have or will not supply the necessary resources for systems to
be designed securely from inception, or when there has been no
historical concern towards security and many systems are already in
production. While it's not a newsflash to anyone in .edu, tight budgets
and timelines often leave security as an afterthought in my experience,
and while I think this needs to change, the resources to make this
happen may not exist. Therefore, my philosophy is that it's better to
perform some type of assessment, ideally before a system goes live, in
order to catch security issues and get them resolved.  I know it may
cost more at this stage, but better to find the problems than not. There
also may be a case where once a system is built, it's not maintained
adequately or is so fragile that no one wants to touch it, or the team
that built it have moved on to new pastures. New vulnerabilities and
attacks emerge, but sometimes the system admins are not making the
required changes and don't keep up with the times. What can be done? A
change in practices, of course, better organizational governance and
policy enforcement. But if that's difficult or very slow to achieve I'd
rather see either an in-house or an outsourced assessment done to find
problems before attackers do, especially for systems such as web
applications. These actions are, of course, part of a package of best
practices.

I'm curious what other .edus are doing with regards to this space. Are
people doing this in-house? Running the usual scanning tools (that do
find low hanging fruit, but miss many issues)? Performing manual
assessment with proxy tools (for webapps), fuzzers, etc? Code review,
security signoff on all projects before they go into production? Is this
work outsourced? Given to the development teams and distributed?
centralized into a security team? How deep do you go with your checks?
Where do these processes fit within your overall priorities? Is it too
expensive to do in-house? If you outsource, what have your experiences
been with services such as Qualys, Whitehat Sentinel, etc. and the
various PCI qualified scanning vendors?

During my consulting work, I have found many security problems that
various scanners missed and I know this is common as there is no
substitute for a skilled analyst. As we all know scanning tools may help
us pluck low-hanging fruit, and stop the people using attack scripts (if
we get there first), but a skilled attacker is a more dangerous thing.
Not to mention that a scanning tool cannot assess business practices
that don't fall into the bits & bytes realm very easily or at all. For
instance, leaving the server room door unlocked, no security camera, no
log review, insecure network design, easily "social engineered",
autoruns enabled, credentials on sticky notes, policies ignored, etc.


Curt Wilson
IT Security Officer & Security Engineer
SIU Carbondale