Re: Application security / penetration testing vendor search

[Sarah Stevens <sarah () STEVENS-TECHNOLOGIES COM>]

Dear Mr. Roberts,

Let me start by stating that my organization is actually a vendor for
the type of work that you are asking about below.  I spend a lot of
time on these discussions, and don't want to falsely represent myself
or my organization as a peer institution.  With that being said, I
would like to share some thoughts on what you should look for in your vendor.

First, you are exactly right.  Anyone can run Nessus or NMAP, gather
a report, and give it to you.  However, once you start running down
through the results, you will quickly realize that much of the
information has to be verified.  I'll give you a perfect
example.  Nessus might give you an indication that you have a SQL
injection attack vulnerability, when in fact you actually do not have
this vulnerability.  Nessus can report a SQL injection vulnerability
based upon a return of an error code when running its signature
scripts.  This error can actually be no more than a poorly written
error catching routine, and not actually a real SQL injection
vulnerability.  Sure, the application returns information stating a
Microsoft error, but the user is not actually immediately able to
return the secrets of your database.  (Of course returning
information from your database that is not supposed to be returned is
the real 'gold' for an attacker executing a SQL injection
attack.)  This is just one example.  Another example would be finding
a vulnerability that is known and expected because there is a
business need for a particular setting on the web server.  In
reality, your seasoned vendor may find that this vulnerability is
mediated by another of your defense in depth mechanisms in your
organization.  So, definitely check up on your vendor.  Make them
share some of their experiences of what they found when running
scans.  Ask about false positive and/or false negative results and
how they handle the various scenarios.

Second, (For folks that read my posts often I will sound like a
broken record at this point.:-)) check out NIST SP800 Series
documents.  OMB A-130 designated NIST as the authority to provide
guidance on securing government information systems.  These documents
are EXTREMELY helpful when choosing what to scan, when to scan, and
even how to set up information security in general.  NIST SP800-42
discusses network security testing.  Now, I know that you have stated
below that you mostly just need help on application specific flaws,
but there is a high level process for this in 800-42.  NIST considers
the "network" as anything that is "networked" into an
organization.  Also, NIST highlights the importance of implementing
security into all phases of the System Development Life Cycle.  So,
once you have a baseline of your current situation, you can integrate
security into further application development during the design phase
instead of waiting until the system has already been developed with
problems.  SP800-37 and SP800-53 discuss more issues relating to
fitting security into the development life cycle.

Third, if you decide to use a vendor, make sure that you have very
clearly defined rules of engagement.  Any organization can be broken
into....but just how far do you want your vendor to go?  Also, once
the vendor has found a vulnerability, you will want to define just
how far the vendor should go in exploiting that
vulnerability.  (i.e.  should the vendor use a SQL injection
vulnerability to inject erroneous data into your database?  What
should the vendor do once they see that they can get in?)

Finally, I truly believe that a "source-code audit" is just a piece
of the puzzle when trying to determine how secure your applications
really are.  You may do a source code audit, but you will definitely
not be able to see every problem that exists from a simple code
review.  New vulnerabilities are discovered every day.  A better plan
of action would be to obtain a full assessment of the exact state of
your application infrastructure at this point, so that you can now
implement security into more layers of the software development life
cycle and the various components of your "network" in the future.

If you have any questions, or would like help preparing your request
for vendor services, let me know.  I'd be glad to help.

Sincerely,

Sarah E Stevens, CISSP, GCIH, CISM
Stevens Technologies, Inc
www.stevens-technologies.com

At 10:50 PM 1/19/2006, Dan Roberts wrote:
> I'm looking for anyone who wouldn't mind sharing some highlights from their
> experiences in picking or using a vendor to perform vulerability
> assessments and penetration testing on web applications.
>
> We've traditionally been good at securing our operating systems and
> networks.. but frankly, the writing has been on the wall for quite some
> time now that patches and firewalls are not adequate protection when public
> facing web applications are riddled with things such as SQL injection
> vulnerabilities and poor authentication schemes.
>
> Recently, we've been given the opportunity to redesign a lot of our
> application development processes.. and you can be sure there will be a lot
> of attention paid to life cycle management, code audits, and vulnerability
> assessments.  The first task at hand is to shore up what we already own.
> The challenge we face is conducting a baseline security review across our
> roughly 40 web applications in very short order.  This is not something we
> intend to develop in-house expertise in; but we can't learn and perform all
> in the short window we've been given.
>
> Enter the need for a vendor to outsource this task to.  The penetration
> testing should be thorough -- performed by an analyst who can dig deeper
> into suspect problem areas, and shed real light on the situation.  That is,
> if the vendor intends to simply run off-the-shelf software and print the
> results for us, I think we're rather capable of doing that ourselves.  It
> should also be focused on application specific security flaws.. we're quite
> handy with Nessus and Nmap type tools, and don't need anyone to confirm for
> us that port 80 is open.
>
> I'm interested in specific vendor recommendations, questions you would have
> asked had you known in advance what a mess you were getting yourself into,
> and input on the effectiveness and efficiency of an pen-test type
> assessment from the outside versus a source code audit.
>
> Thank you in advance,
>
> Dan Roberts
> Office of Security and Compliance
> Information Services Division
> Kent State University
>
> 330-672-5373
> ddrobert () kent edu