Re: Certifications: Not worth the paper they are printed on?

[Jon Kibler <Jon.Kibler () aset com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Mario,

Please see my embedded comments.

Mario Platt wrote:
> Hey Jon,
>
> What you point out in your e-mail are "good" concerns towards our
> industry, in general. But many things that you point out are, in my
> opinion, wrong.
> It is true that most people can pass most certifications not having
> real world experience,and I say that is a good thing, considering SOME
> certifications.

If certifications are supposed to be a means of verifying expertise,
then how can you have expertise in a subject if you have no real
experience working in that subject area?

> As in any certification, there is a good way to take it and a bad way
> to take it. Every certification that I take, I take it seriously and
> don't limit myself to studying only the minimum required to pass the
> exam. If someone else does it, and I know many people do, good for
> them. I believe certifications can open a door for you in the market,
> but if you are not a good professional and don't show the skills that
> you are "supposed" to have from those certifications, it's up for your
> employer to keep you or send you away.

But, the problem here is that once you get your foot in the door, at
many employers it is so difficult to get rid of someone that they simply
have to put up with under performance. Companies are so afraid of being
sued that they keep incompetent employees just to avoid being sued.

> Even though there are many things that could change, and make the
> certiication industry better, I think that what really needs to be
> done is manager education, as to what skills people are supposed to
> have.

But that is the problem. Most managers (etc.) are clueless as to how to
screen people for new positions where no one on staff has any expertise
in the subject area. Security is the perfect example. I see company
after company that have had serious security issues and have had to hire
someone to "fix security" because no one on staff knows enough to fix
the problems. So, what do they do? They hire a recruiter to find some
one that appears to be qualified. How does the recruiter judge
qualifications? Well, certifications top the list.

Within the past few months I have seen one organization that hired a
contract security organization to manage their security. When I looked
at what this company was supposed to be doing (monitoring logs,
maintaining patches, monitoring network traffic, etc.), I found that
they were only partially meeting one of over a dozen contractual
obligations. The client, lacking internal expertise, was clueless to
have figured out on their own that they were being taken for a ride by a
major, allegedly reputable, consulting firm that they had used for
nearly a decade in other capacities before adding the monitoring contract.

In another organization, they had hired a "security manager" to manage
network security. Among her assignments were to set up network firewalls
between WAN nodes. When I audited the organization and found the
firewalls had not been configured, I found that no one in the
organization had the expertise to tell that the firewalls were not
working and the security manager admitted she had not configured them
because she really was not comfortable doing so and knew if she did it
wrong it would break a bunch of stuff and she would look bad. So, to
make management think that everything was working fine, she simply
generated weekly reports showing how much Internet traffic had been
blocked by routers and called it her "firewall report."

So, as much as we would like to be able to say that managers need to
know what to look for when interviewing for a given technical skill,
that is not reality in most organizations until you reach the Fortune
1000 companies. IT shops are simply too small and tend to have only one
or maybe two people with expertise in any given critical area.

Managers simply cannot be expected to have the technical expertise to
evaluate all potential employees. That is why certifications were
developed. That is also why it is important that the certifications are
meaningful representations of an individual's real working skill set.


> 100+ certifications, in my opinion, is ridiculous because I don't know
> anyone that could be "that good" in so many things (as I don't believe
> there are that many certifications on the same subjects), but having
> some certifications, today, is indispensable for your employer,
> typically.

> Certifications are supposed to give you "thought
> processes", and general/specific knowledge about a subject. they are
> not supposed to give you GURU status on anything.

Ah, but that is NOT how they are being marketed and that is DEFINITELY
not how the certified individuals are using the fact they are certified.
Instead, we have both the implicit and explicit perception that "I am
certified, therefore I am an expert in this area -- with my
certification being the proof of my expertise." Certification marketers
are just as guilty of this hype as are the certification holders.

> The employers are the ones that need to realize that today, you can
> find most of the answers to the certification exams online, and
> because of that you can't tell just by looking at the certification
> part of a resume, that THAT is the person you want in your company. I
> don't think this is "broken", as you said. I think that this is the
> only way that this could happen.

But, if we had certifications that truly evaluated someone's ability to
DO something, then certifications would be meaningful. Since
certifications are often the only means an employer has to judge
someone's qualifications, the certifications need to reflect ability,
not just knowledge.

I recently worked with a client to find a person with a particular
security and networking skill set. We waded through about two dozen
resumes to pick out the top 5 candidates on paper. When those resumes
were given to HR to verify, 4 of the 5 candidates were found to have
fraudulent resumes: degrees that they did not have, trying to take
credit for work others had done at a previous employer, etc.

The resume review process and trying to validate someone's experience on
their resume is also made more difficult by former employer's desire to
not be sued. Often, you can only get verification that someone either
did or did not work at a given company. Most companies are not even
willing to say if that person was potentially eligible to be rehired
should they apply for a job at that former employer in the future.

It is unfortunate, but resume fraud has become so pervasive that a
resume must be simply viewed as "marketing hype" for an individual, with
any truth in the resume to be determined in the interview process. If a
prospective employer does not have the expertise to adequately screen
candidates on a technical basis, they are left to rely on the
candidate's certifications as the only means to verify the potential
employee has half a clue about the job they may be hired to perform.

> You can demand that individuals need to have at least X years of
> experience in order to take an exam, but how can a vendor confirm that
> ? You have thousands of companies, and if they want someone to take a
> certification path, the person could be there for a week and they can
> write a paper saying that he works has been working there for 10
> years. There is no way a vendor can guarantee that.

CISSP requires proof of work experience. Others should do the same.

If you make a test candidate's proof of experience a legal affidavit,
and it is notarized, then any fraudulent statement on that affidavit
become perjury -- a felony.

> I don't think that all certifications should have an expiration date.
> Many certifications make you knowledgeable on a specific version of a
> product. Why should I lose that certification if I know how to work on
> THAT version of a product, and work everyday with it ?

But, a product version IS an expiration date. The certification is valid
for only that version of the product. When that version dies, so does
the certification.

> Can you please explain to me how do you make that assumption that
> everyone attending DEFCON is knowledgeable in "whatever" ? I believe
> that makes no sense.

Context here, please! I said that if someone is going to teach hacking,
they should at least attend a hacker conference to keep up to date on
the latest information relevant to what it is they are teaching!

I never said that everyone attending Defcon was knowledgeable. But, I
did say that if you are teaching hacking, you should be BOTH
knowledgeable AND experienced as a hacker!

> > Until certifications can become a meaningful means of verifying a
> > claimed level of experience and expertise, they shall remain not worth
> > the paper they are printed
>
> They can't and shouldn't, in my opinion. I've met people with 3/4
> years experience with more knowledge than someone who's been in the
> industry for over 10, so why should they be "held back" just because
> they are not here for that many time ?

But, I also said that one of the purposes of a certification is to
differentiate between a real "10 years of experience", and someone who
has worked 10 years in the industry but stopped learning anything new as
soon as they got their job.

I think for most entry level certifications, at least 2 years of
experience should be required. I don't care how good you are, if you
have not done something on a day-to-day basis for 2 years, you have not
seen enough ways things break to have a real understanding of your area
specialization. That is why doctors have 2 to 3 years of residency
before private practice -- in med school they learned (we hope!) what TO
DO, and residency is where they learn what NOT TO DO! We need the same
"maturation time" (and this accumulation of negative knowledge) in any
area of expertise before we can put our stamp of approval
(certification) on an individual, saying that they have more than 1/2 a
clue what it is that they are doing.


I hope I have clarified some of my points!

Jon
- --
Jon R. Kibler
Chief Technical Officer
Advanced Systems Engineering Technology, Inc.
Charleston, SC  USA
o: 843-849-8214
c: 843-224-2494
s: 843-564-4224
http://www.linkedin.com/in/jonrkibler

My PGP Fingerprint is:
BAA2 1F2C 5543 5D25 4636 A392 515C 5045 CF39 4253


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkjpKy8ACgkQUVxQRc85QlOoVwCglBaMGBcZl1/qIBjP0q2VCZS6
voIAni5l9oEp7MsmZz+FXrrS+1jD93EQ
=jykp
-----END PGP SIGNATURE-----




==================================================
Filtered by: TRUSTEM.COM's Email Filtering Service
http://www.trustem.com/
No Spam. No Viruses. Just Good Clean Email.


------------------------------------------------------------------------
This list is sponsored by: Cenzic

Top 5 Common Mistakes in 
Securing Web Applications
Get 45 Min Video and PPT Slides

www.cenzic.com/landing/securityfocus/hackinar
------------------------------------------------------------------------