Penetration Testing mailing list archives
Re: Certifications: Not worth the paper they are printed on?
From: "Mario Platt" <mplatt () gmail com>
Date: Sun, 5 Oct 2008 20:57:18 +0100
Hey Jon, What you point out in your e-mail are "good" concerns towards our industry, in general. But many things that you point out are, in my opinion, wrong. It is true that most people can pass most certifications not having real world experience,and I say that is a good thing, considering SOME certifications. As in any certification, there is a good way to take it and a bad way to take it. Every certification that I take, I take it seriously and don't limit myself to studying only the minimum required to pass the exam. If someone else does it, and I know many people do, good for them. I believe certifications can open a door for you in the market, but if you are not a good professional and don't show the skills that you are "supposed" to have from those certifications, it's up for your employer to keep you or send you away. Even though there are many things that could change, and make the certiication industry better, I think that what really needs to be done is manager education, as to what skills people are supposed to have. 100+ certifications, in my opinion, is ridiculous because I don't know anyone that could be "that good" in so many things (as I don't believe there are that many certifications on the same subjects), but having some certifications, today, is indispensable for your employer, typically. Certifications are supposed to give you "thought processes", and general/specific knowledge about a subject. they are not supposed to give you GURU status on anything. The employers are the ones that need to realize that today, you can find most of the answers to the certification exams online, and because of that you can't tell just by looking at the certification part of a resume, that THAT is the person you want in your company. I don't think this is "broken", as you said. I think that this is the only way that this could happen. You can demand that individuals need to have at least X years of experience in order to take an exam, but how can a vendor confirm that ? You have thousands of companies, and if they want someone to take a certification path, the person could be there for a week and they can write a paper saying that he works has been working there for 10 years. There is no way a vendor can guarantee that. I don't think that all certifications should have an expiration date. Many certifications make you knowledgeable on a specific version of a product. Why should I lose that certification if I know how to work on THAT version of a product, and work everyday with it ? Can you please explain to me how do you make that assumption that everyone attending DEFCON is knowledgeable in "whatever" ? I believe that makes no sense.
Until certifications can become a meaningful means of verifying a claimed level of experience and expertise, they shall remain not worth the paper they are printed
They can't and shouldn't, in my opinion. I've met people with 3/4 years experience with more knowledge than someone who's been in the industry for over 10, so why should they be "held back" just because they are not here for that many time ? Just my 2 cents... Best regards Mario Platt On Sun, Oct 5, 2008 at 7:15 PM, Jon Kibler <Jon.Kibler () aset com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 All, Yesterday I was reading a blog where someone with no security experience whatsoever was grousing that they flunked the Security+ exam. The blogger also claimed to have over 100 certifications. In my opinion, that many certifications undoubtedly qualifies this blogger to be the Poster Boy for everything that is wrong with the certification process. I do not know of anyone who has the real world experience to pass 100+ certification exams based only upon their experience. The fact that someone can pass a certification exam WITHOUT ANY EXPERIENCE clearly illustrates something is critically wrong with our industry's certification process. (MCSE: Must Call Someone Experienced!) The certification process today is utterly and completely broken. The single biggest problem that I see with the certification industry is the scarcity of "real world" certifications -- those certifications that cannot be passed by book knowledge alone -- certifications that require hands-on real-world experience to pass, such as the RHCE, CCIE, or any of the GIAC Gold certifications. All certifications should be as rigorous as these and similar certifications that reflect one's ability to do real work in the area in which they are certified. In my humble opinion, most certifications today are not worth the paper they are printed on. Certifications were originally conceived as a means to help weed out fictitious resumes, or to verify that someone claiming to have "10 years of experience" is not someone who really has "the equivalent of one year of experience, times ten." However, the fact that so many certifications are so lame that anyone can buy a book, memorize it, and take and pass an exam, shows how critically broken is the certifications process. Most certifications today do not show that you are capable of DOING anything except memorizing mostly useless and dated facts. Certifications have gone from something potentially useful and meaningful to being the equivalent of Country Club Dues. It has become the price of admission to join a certain group of people in the workplace. Just like your ability to pay your country club dues does not say anything about your ability to play golf, certifications say nothing about your ability to do the work associated with the certification. We need to change certifications from being country club dues to being more like PGA tour qualifications. The entire certification process needs to change. Certifications must once again reflect an individual's ability to DO something, verses their ability to memorize. When someone presents a certification, an employer needs to have some confidence that the prospective employee can actually do the job in the real world. What needs to change? At least four things immediately come to mind: 1) Before taking a certification exam, you must be able to demonstrate an auditable degree of associated work experience. For example, the new Security+ certification calls for a minimum of 2 years of day-to-day security experience as a recommended prerequisite. Well, it should be made a REQUIREMENT that you MUST HAVE at least 2 years of experience doing day-to-day security work before you are allowed to sit for the exam. 2) Exams must be changed from being fact-based to become experience-based. It should not be possible to simply read books and pass an exam. For example, the Security+ exam should include questions that only a security practitioner would be able to answer. It should include packet captures and ask for an interpretation. It should require you to be able to verify a digital signature. It should present log files and ask you to identify how the system was compromised. Etc. Real world experience-based questions should be an integral part of each exam's questions. It should not be possible to pass the exam without the required hands-on experience. 3) Certifications must have an expiration date. Knowledge in every area of technology is transient in nature. Certifications must reflect that they are based on the qualifications to do a job at a particular point in time, and that those qualifications will change over time. As I stated previously, the initial certification should require auditable work experience. Recertification should require not only demonstrated continued work experience, it should also require CEUs/CPEs to maintain the certification. In fact, continuing education should be made an annual requirement to maintain certifications between recertifications. 4) Instructors teaching certification courses *MUST* have demonstrable real world work experience before being deemed qualified to teach the certification course. Probably the two certifications with the greatest "Instructor Qualification Laugh Factor" are the EC-Council's CEH and CHFI courses. The majority of instructors that I have met that teach either of these two courses have NEVER done ANY real work in either associated profession. -- How can an instructor properly convey to students the real thought processes of a hacker, if they themselves have not performed dozens of successful real world penetration tests? -- How can an instructor properly convey to students all that they need to know about forensics, if they themselves have never performed a real world forensics examination, and prepared and presented evidence in court? -- It is simply not possible to study, get a certification, and teach these (and similar) courses without the instructor and ed center doing an extreme disservice to their students. Instructors should be required to not only have the certification, but they must have real world work experience actually doing what they are teaching. -- Instructors should also be required to maintain additional CEUs/CPEs beyond those required to maintain certification. Attending two relevant conferences a year should be mandatory. (I would bet that most CEH instructors have never even been to Defcon! How many CHFI instructors have ever attended TechnoForensics? I bet almost none have!) Similar qualifications and continuing education needs to be mandated of all instructors teaching in any area of technology. Perhaps another analogy would help clarify my concerns. Would you hire a pilot for your corporate jet that only has a certificate saying that they had passed flight school ground training? Someone that had no actual experience as a pilot? Would you want this same person teaching other wannabe pilots? I would hope not! However, that is the situation we find ourselves in with technology certifications. We are getting hordes of people that simply "pass ground school" and now claim to be "capable of flying a 747." Still worse, the majority of our instructors for technology certifications have only "passed ground school", but are using that as the basis to hang out their shingle claiming that they can teach others to fly, when they themselves have never even seen the inside of the cockpit of an airplane, not less ever actually having piloted a real aircraft. Until certifications can become a meaningful means of verifying a claimed level of experience and expertise, they shall remain not worth the paper they are printed on. In the meantime, we in the industry need to educate our managers, and our training and HR departments as to what certifications are meaningful and which ones are not. At the same time, we need to be teaching them what certifications are appropriate for a given job skill. For example, I see CISSP mandated for numerous jobs (such as penetration tester) where other more appropriate certifications should be used instead. But, because CISSP is thought to be the ultimate certification in security, they think that "one size fits all" security positions. We need help change that thought process! Jon Kibler - -- Jon R. Kibler Chief Technical Officer Advanced Systems Engineering Technology, Inc. Charleston, SC USA o: 843-849-8214 c: 843-224-2494 s: 843-564-4224 http://www.linkedin.com/in/jonrkibler My PGP Fingerprint is: BAA2 1F2C 5543 5D25 4636 A392 515C 5045 CF39 4253 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkjpBCsACgkQUVxQRc85QlOi4gCglvr/TnrMop6vn2I+1dzSgTbY m+0AniDUj/eM0o28f2vKRgdpV9Suhx57 =pcU9 -----END PGP SIGNATURE----- ================================================== Filtered by: TRUSTEM.COM's Email Filtering Service http://www.trustem.com/ No Spam. No Viruses. Just Good Clean Email. ------------------------------------------------------------------------ This list is sponsored by: Cenzic Top 5 Common Mistakes in Securing Web Applications Get 45 Min Video and PPT Slides www.cenzic.com/landing/securityfocus/hackinar ------------------------------------------------------------------------
------------------------------------------------------------------------ This list is sponsored by: Cenzic Top 5 Common Mistakes in Securing Web Applications Get 45 Min Video and PPT Slides www.cenzic.com/landing/securityfocus/hackinar ------------------------------------------------------------------------
Current thread:
- Certifications: Not worth the paper they are printed on? Jon Kibler (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? Mario Platt (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? Jon Kibler (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? Terry Cutler (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? David Howe (Oct 06)
- Re: Certifications: Not worth the paper they are printed on? Jay D. Dyson (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? brendan (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? Jay D. Dyson (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? David Howe (Oct 06)
- Re: Certifications: Not worth the paper they are printed on? Jimmy Brokaw (Oct 06)
- Re: Certifications: Not worth the paper they are printed on? brendan (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? Jon Kibler (Oct 05)
- Re: Certifications: Not worth the paper they are printed on? John Mason Jr (Oct 06)
- Re: Certifications: Not worth the paper they are printed on? Mario Platt (Oct 05)