Penetration Testing mailing list archives
RE: Vulnerability Assessment vs. PenTest
From: "David M. Zendzian" <dmz () dmzs com>
Date: Wed, 9 Aug 2006 17:11:59 -0700
One thought on this, btw great writeup. An audit makes no assumptions about the future. It is a view at the moment of the audit which can and will most likeley change the moment after the auditor leaves. David -----Original Message----- From: "Craig Wright" <cwright () bdosyd com au> To: stylewar () cox net; "Christine Kronberg" <seeker () shalla de>; "Arkem Paul" <bob () mornmist com> Cc: pen-test () securityfocus com Sent: 8/8/06 9:35 PM Subject: RE: Vulnerability Assessment vs. PenTest To add a little to this debate. First, there are two types of Audit, internal and external. An audit, consisting of an evaluation of an organisation's systems processes and controls, is performed against the set standard or documented process. Audits are designed to provide an independent assessment through a qualified independent assessment of representations about the system or process. An audit may also provide a gap analysis of the operating effectiveness of the internal controls. An audit differs from an inspection in that an audit makes representations about likely future results. An inspection evaluates past results. Or an audit to be valid it must be conducted according to accepted principles. In this, the audit team and individual auditors must be certified and qualified for the engagement. Numerous "audits" are provided without certification, these however are qualified reviews. A penetration test is an attempt to bypass controls and gain access to a one system. The goal of the penetration test is to prove the that the system may be compromised. A penetration test does not assess the relative control strength nor the system or processes deployed, rather, it is a "red teaming" styled exercise designed to prove illicit access. The real strength of a penetration test is marketing the need to improve controls to internal management. A penetration test is of limited value in the greater scheme of a systems information security due to the restricted nature of the test and the lack of inclusion of many key controls. A vulnerability assessment is an assessment and gap analysis of a site's or a system's control strengths. A vulnerability assessment is a risk based process. The process involves the identification and classification of the primary vulnerabilities which may impact the system. Often, methodologies such as fault tree analysis end cause consequence analysis are employed in this review. Both vulnerability assessments and penetration tests may be conducted as a white box or black box analysis. A black box analysis is instigated with little or no knowledge of the system being tested. A white box analysis is conducted for knowledge of the system. A vulnerability assessment is a critical component of any threat risk assessment. Following the vulnerability assessment and impact analysis is conducted and used in conjunction with a threat report to provide for an estimation of the organisation's risk to selected attack vectors. External audits are conducted (or at least should be) by independent parties no rights or ability to alter or update the system. Internal audits involve a feedback process where the auditor may not only audit the system but also potentially provide advice in a limited fashion. And external auditor is precluded from advising their client. They are limited to reporting any control gaps and leading the client to a source of accepted principles. The common perception that running an automated scanner such as Nessus or one of its commercial cohorts is in itself a vulnerability or penetration test is false. Most of the so-called penetration tests that are provided are no more than a system scan using tools. A penetration test it correctly provided will attempt the use of various methodologies to bypass controls. In some instances this may involve the creation of new or novel scripts/programs. The issue is not that many people commonly use the words interchangeably but that so-called professionals fail to differentiate the terms. Of particular concern is the use of audit and the designation auditor. This is as these terms are often restricted in code. This is that most jurisdictions have statutory requirements surrounding their use and application. Information security systems provide many of the functions that construct a control system. Of particular concern are controls that limit access to accounting and financial records. This includes records held by systems that provide an e-commerce transaction path. In many jurisdictions it is an offence to sign off an audit report when you are not a certified auditor. Traditionally the path around this has been not to call the process of testing the system and audit, but rather to call it an agreed procedures review. An agreed procedures review or simply a review is an analysis of controls performed against an agreed process. Some example's of an audit include SAS 70 (part 1 or 2) audits, ISO 9001,17799:2/27001 certification audits, HIPPA audits. There are many different types of audits and many standards that an audit may be applied against. There are various processes and procedures used to provide vulnerability assessments and threat risk analysis. Standards such as AS/NZS 4360:2006 I commonly mandated by government organisations. Penetration testing, if done correctly, may provide some value in its free-form approach. When correctly implemented, a penetration test adds a level of uncertainty to the testing. The benefit of this uncertainty is that it might uncover potential flaws in the system or controls that had not been taken into account when designing the control system. To be of value, a penetration test must needs to do more than scan a system. It needs to do something novel and unexpected. There is little similarity between a penetration test, vulnerability assessment, risk assessment or audit. The lack of understanding of these differences impedes the implementation of effective security controls. Bus to finish, 'Stylewar' is correct in stating that "an audit must follow a rigorous program...". Christine's appraisal of a vulnerability assessment would more correctly be termed as a controls assessment. A controls assessment may also be known as a security controls review. As for the need to develop a structured taxonomy (naming system), there is already one in existence. None of these terms or services is new. All these services have been provided for as long as computers have been used by business and government. They were definitely employed as far back as the 70s. Regards, Craig -----Original Message----- From: StyleWar [mailto:stylewar () cox net] Sent: Wednesday, 9 August 2006 3:19 AM To: 'Christine Kronberg'; 'Arkem Paul' Cc: pen-test () securityfocus com Subject: RE: Vulnerability Assessment vs. PenTest Point of fact that an audit must follow a rigorous program, and has a set of documentation and traceability requirements with it that an 'assessment' does not. They are 'approximate' in the hands of a well disciplined assessment team - but I would stop a hair short of calling them equal.. - StyleWar "Ancora Imparo"
-----Original Message----- From: Christine Kronberg [mailto:seeker () shalla de] > Sent: Sunday, August 06, 2006 11:54 AM To: Arkem Paul Cc: pen-test () securityfocus com Subject: Re: Vulnerability Assessment vs. PenTestOn Sun, 6 Aug 2006, Arkem Paul wrote:A Vulnerability Assessment should be a comprehensive look > from policy > > and procedures to implementation of security in the network > and should > > include such things as patch management, virus protection, user > > education, SOE hardening, infrastructure configuration, etc. So basicly an assessment is equal to an audit? The > description aboveis what I usually expect from someone doing an audit. A vulnerability assessment I tend to understand in terms > of investigating a specific application (in far more detail than a > penetration test).There are a couple of term mixed every now and again (like someoneelse just stated: funny that we professionals don't come > up with _one_ definition):AuditSecurity Scan Security Assessment Vulnerability Assessment Penetration TestDid I miss one? Cheers, Christine Kronberg.
Liability limited by a scheme approved under Professional Standards Legislation in respect of matters arising within those States and Territories of Australia where such legislation exists. DISCLAIMER The information contained in this email and any attachments is confidential If you are not the intended recipient, you must not use or disclose the information. If you have received this email in error, please inform us promptly by reply email or by telephoning +61 2 9286 5555. Please delete the email and destroy any printed copy. Any views expressed in this message are those of the individual sender. You may not rely on this message as advice unless it has been electronically signed by a Partner of BDO or it is subsequently confirmed by letter or fax signed by a Partner of BDO. BDO accepts no liability for any damage caused by this email or its attachments due to viruses, interference, interception, corruption or unauthorised access. ------------------------------------------------------------------------------ This List Sponsored by: Cenzic Concerned about Web Application Security? Why not go with the #1 solution - Cenzic, the only one to win the Analyst's Choice Award from eWeek. As attacks through web applications continue to rise, you need to proactively protect your applications from hackers. Cenzic has the most comprehensive solutions to meet your application security penetration testing and vulnerability management needs. You have an option to go with a managed service (Cenzic ClickToSecure) or an enterprise software (Cenzic Hailstorm). Download FREE whitepaper on how a managed service can help you: http://www.cenzic.com/news_events/wpappsec.php And, now for a limited time we can do a FREE audit for you to confirm your results from other product. Contact us at request () cenzic com for details. ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ This List Sponsored by: Cenzic Concerned about Web Application Security? Why not go with the #1 solution - Cenzic, the only one to win the Analyst's Choice Award from eWeek. As attacks through web applications continue to rise, you need to proactively protect your applications from hackers. Cenzic has the most comprehensive solutions to meet your application security penetration testing and vulnerability management needs. You have an option to go with a managed service (Cenzic ClickToSecure) or an enterprise software (Cenzic Hailstorm). Download FREE whitepaper on how a managed service can help you: http://www.cenzic.com/news_events/wpappsec.php And, now for a limited time we can do a FREE audit for you to confirm your results from other product. Contact us at request () cenzic com for details. ------------------------------------------------------------------------------
Current thread:
- Re: Vulnerability Assessment vs. PenTest, (continued)
- Re: Vulnerability Assessment vs. PenTest Arkem Paul (Aug 05)
- Re: Vulnerability Assessment vs. PenTest Christine Kronberg (Aug 06)
- RE: Vulnerability Assessment vs. PenTest Richard Feist (Aug 07)
- Re: Vulnerability Assessment vs. PenTest xelerated (Aug 07)
- Re: Vulnerability Assessment vs. PenTest Magdelin Tey (Aug 07)
- RE: Vulnerability Assessment vs. PenTest StyleWar (Aug 08)
- RE: Vulnerability Assessment vs. PenTest Omar A. Herrera (Aug 07)
- Port Listening Chris Esezobor (Aug 10)
- RE: Port Listening Luke Walsh (Aug 10)