Re: Value and use of penetration testing and vulnerability assessment in .edu

[Heather Flanagan <heatherf () STANFORD EDU>]

On Jan 17, 2008, at 7:54 AM, Curt Wilson wrote:

> Dear Educause security community -
>
> In addition to my work at the university here, I have done work as
> a consultant and in that capacity, performed many vulnerability
> assessments and some penetration tests. I know that they are
> sometimes over-hyped and are not the solution to all of our
> security issues like some vendors would have us believe, however I
> have seen significant value delivered through these practices,
> especially when an organization does not have or will not supply
> the necessary resources for systems to be designed securely from
> inception, or when there has been no historical concern towards
> security and many systems are already in production. While it's not
> a newsflash to anyone in .edu, tight budgets and timelines often
> leave security as an afterthought in my experience, and while I
> think this needs to change, the resources to make this happen may
> not exist. Therefore, my philosophy is that it's better to perform
> some type of assessment, ideally before a system goes live, in
> order to catch security issues and get them resolved.  I know it
> may cost more at this stage, but better to find the problems than
> not. There also may be a case where once a system is built, it's
> not maintained adequately or is so fragile that no one wants to
> touch it, or the team that built it have moved on to new pastures.
> New vulnerabilities and attacks emerge, but sometimes the system
> admins are not making the required changes and don't keep up with
> the times. What can be done? A change in practices, of course,
> better organizational governance and policy enforcement. But if
> that's difficult or very slow to achieve I'd rather see either an
> in-house or an outsourced assessment done to find problems before
> attackers do, especially for systems such as web applications.
> These actions are, of course, part of a package of best practices.
>
> I'm curious what other .edus are doing with regards to this space.
> Are people doing this in-house? Running the usual scanning tools
> (that do find low hanging fruit, but miss many issues)? Performing
> manual assessment with proxy tools (for webapps), fuzzers, etc?
> Code review, security signoff on all projects before they go into
> production? Is this work outsourced? Given to the development teams
> and distributed? centralized into a security team? How deep do you
> go with your checks? Where do these processes fit within your
> overall priorities? Is it too expensive to do in-house? If you
> outsource, what have your experiences been with services such as
> Qualys, Whitehat Sentinel, etc. and the various PCI qualified
> scanning vendors?
>
> During my consulting work, I have found many security problems that
> various scanners missed and I know this is common as there is no
> substitute for a skilled analyst. As we all know scanning tools may
> help us pluck low-hanging fruit, and stop the people using attack
> scripts (if we get there first), but a skilled attacker is a more
> dangerous thing. Not to mention that a scanning tool cannot assess
> business practices that don't fall into the bits & bytes realm very
> easily or at all. For instance, leaving the server room door
> unlocked, no security camera, no log review, insecure network
> design, easily "social engineered", autoruns enabled, credentials
> on sticky notes, policies ignored, etc.

Hi Curt -

Funny you should ask - I just forwarded a third-party pen. test
report to our Information Security Office!  Central IT at Stanford is
using third parties for penetration tests simply to validate the work
we're doing ourselves.  We are doing bi-weekly scans from internal
networks, monthly scans "behind" firewalls to see what people are
assuming the firewall will hide, and quarterly third party pen. tests
to verify our methodology is good and we haven't missed anything.
This is of servers we manage, tho' we're also considering turning
this into a service that departments can use if they don't have the
resources to set up their own internal scanning.

The scanning we're doing so far is purely network scanning.  We're
not working through any social engineering attacks at this time, not
from outside anyway.

The Information Security Office has a bigger role for the campus
overall in terms of education and policy.  Our security efforts are
to make sure we're in compliance with those policies.

I hope that helps!

Heather Flanagan
Director, System Administration
heatherf () stanford edu