Penetration Testing mailing list archives
Re: OSCP
From: "Andre Gironda" <andreg () gmail com>
Date: Fri, 12 Dec 2008 12:11:39 -0700
On Fri, Dec 12, 2008 at 4:32 AM, <christopher.riley () r-it at> wrote:
If you actually do real security, OSCP is a bullshit cert just like CEH, CNOP, SCNA, GSE, et al. Anything SANS or ISC2 is crap. All of these certs mean absolutely nothing if you dig deep into the actual meaning behind them.Just out of interest, have you actually done any of the training that you think is "crap". Having been through some of the training you mention (some of it good, some of it bad) I would disagree with your broad assumption. Have you ever actually met somebody with a GSE for example. It's not an easy qualification to achieve. The courses you mentioned are designed to educate in a specific area and not be a wide ranging (but thin on information) course like the CISSP (and other) training appears to be.
Individuals are individuals. Some are better than others, but nobody is cookie cutter in our industry - let alone most other industries. Speaking to the material itself, I have read and maintain current copies of all of this training material and I think I'm fairly familiar with most of the people who write the training material. CNOP is only a certification criteria, but they set out a pretty specific goal and timeline. Even GSE is fairly well known, and the training is just a collection of the total sum of SANS training.
The question I have is what do you class as "real security". I work as a penetration tester and can say from my side that CEH is not a good course (in regards to preparing you for working in the ethical hacking field), however the SANS classes (SEC:560, SEC:504, SEC:709, SEC:542, etc...) do provide a great deal of information. No training course or certificate is going to guarantee a person is fit for the job, but dismissing them all as "crap" is in my opinion unfounded.
I suggest that you read the full OSSTMM 3.0 for "real" aka "operational security". Also worth checking out would be NIST SP800-30, NSA IAM/IEM/RTM, DOD DIACAP, and Andrew Jaquith's SecurityMetrics book/blog/mailing-list. There have been interesting threads on the scadasec mailing-list lately as well. I have read/viewed/listened-to SANS 502, 503, 504, 505, 508, 517, and 617 training material and know some that have attended those classes. There are descriptions and outlines (more detailed than what is available from SANS) for 560 on some wikis and blogs in various places. I am positive that 542 is a joke/crap because I am a regular web application blogger and guru. Here is a good summary of 709: http://c22blog.wordpress.com/2008/12/10/sans-sec709-developing-exploits-for-penetration-testers-day-2/ The outline for the 4-day course is up http://www.sans.org/sans2009/description.php?tid=2717 I think all of these courses are interesting to some degree, but how do they help with operational security? What does the certification say about the person? Why not just list the training classes you have attended on your resume, instead of purporting to be capable of "securing" something instead of being only knowledgeable about "demonstrating use of the knowledge demonstrated in 1-N class(es) or 1-N exam(s)"? Finally, what is the correlation between offensive security skill/research and operational security? Just because a company gets hit with one zero-day that owns a partial part of the infrastructure doesn't indicate that anything serious (i.e. PII or confidential data) has been breached. It doesn't talk about access controls, auditability, incident response, et al. The OSSTMM 3.0 lists only 10 controls (5 interactive and 5 process) for a security posture, but the RAV and STAR calculations are worth a serious look. The NSA IEM has IPP (rating system) and DOD DIACAP has a scorecard. These say something about the security of something (network/infrastructure, or even a particular system in the case of DIACAP). Over the many years I've seen people talk about certifications -- most admit that they really like CCIE because you have to demonstrate something and "It's not an easy qualification to achieve" (like some say about GSE). While there are even fewer GSEs than CCIEs (even when CCIE was as mature -- as many years young -- as GSE), I always found CCIEs to be completely clueless. It was common unspoken practice that Cisco-employee CCIE examiners/trainers would have to be certified (Train the Trainer, or TTT). This material leaked inside Cisco (as well as outside the organization) and you had tons of paper-CCIEs. How does leaked information lead to success with a lab based exam, you ask? Because a lab is a sample set of processes that are repeated over and over. It's common practice today to download a pass4sure or actualtests PDF guide that contains all of the possible answers for any IT exam (including CISSP, CISA, and many many others). Statistically, if you know the exam's potential maximum amount of questions and have all of the potential answers, you can find a happy medium of how many you have to memorize at minimum in order to meet the pass rate for that exam. This works equally in a lab environment as it does on a computer-based test package. I have found that the best way to fix this error is not only to have multiple exams, but to also have very long exams, with an extremely large amount of potential questions that are chosen at random. However, at this point, a prospecting certification organization should simply open up their questions AND answers. i.e. "In order to certify for the OPCP, you must pass 10 exams, each consisting of 2000 in-exam questions, taken from a total of 12,000 potential question-answer combinations". This prevents actualtests/pass4sure, and it also evens the playing field. Another problem is the design of the questions and answers. You'd have to look at the structure of a medical or law degree program to come to something close to what we'd need for our field (i.e. "operational security"). This is actually the intent of the OWASP OPCP project. It is not a certification, nor does it provide any classes or material for training. It is simply an open set of questions and answers that can be used. If ISC2 would like to steal these ideas (which aren't even really mine), that's fine. They already started to snake in the appsec industry with CSSLP_WhitePaper_3.pdf (hrmn... is that a leaked file?). Will SANS move to this sort of model? I don't know the answers to these questions. However, there comes a time when this madness must cease to continue and we should all work together to stop the hamster wheel of pain. Cheers, Andre ------------------------------------------------------------------------ This list is sponsored by: Cenzic Security Trends Report from Cenzic Stay Ahead of the Hacker Curve! Get the latest Q2 2008 Trends Report now www.cenzic.com/landing/trends-report ------------------------------------------------------------------------