Re: ICSA

[David LeBlanc <dleblanc () mindspring com>]

At 10:21 PM 1/18/99 +0000, Crispin Cowan wrote:
> David LeBlanc wrote:
>
> > [warning - USENET flame/quibble mode ON]
> >
> > At 11:18 AM 1/14/99 -0500, Richard Reiner wrote:
> >
> > > In brief: Any organization which denominates itself an "Association", but
> > > which is in reality a private, for-profit company, seems to me to lose all
> > > credibility.  (Were this not a public forum, I would use far stronger
terms
> > > to say what I think of this practice.)
> >
> > You say "lose all credibility".  This is pure nonsense, and as a fellow
> > Ph.D., I am appalled at your use of such broad terminology.
>
> I dunnow, I found his use of the word pretty precise.  "lose all credibility"
> as in "demonstratably willing to mis-represent the truth for their own
> purposes."  Works for me.

What did they misrepresent?  I'm not aware of them misrepresenting any
vendor passing the tests or not.  Again, if some specific evidence of
wrongdoing were cited, I'd find the accusation a lot more credible.
 
> > Further, I'd be interested in why you think that being a private,
> > for-profit company makes them any less or more likely to produce unreliable
> > or biased information?

> Being a private for-profit company that deliberately gives the appearance of
> being a non-profit association is different from being a private for-profit
> company and being stand-up about it.

This really doesn't matter much to me - I'm looking at what the bottom line
is here.  I'd tend to agree that they should be straightforward about the
fact that they are a for-profit organization, but I don't see any
connection between that and what these tests mean to network security.

> >  Let's
> > get realistic here - unless you're willing to go get funding for a
> > competing organization and get all the vendors to use your process, then it
> > is either ICSA or believe the vendors (yeah, right).

> I would rather have nothing at all than an evaluator that claims to be
> objective and independent but isn't.  That's worse than "believe the
vendors",
> that's renting the vendors extra credibility for a price.

I could make a strong argument that being for-profit would tend to make
them more independent, not less.  As to whether they are objective, I still
haven't seen anyone post substantiated evidence of specific wrong-doing.  I
have seen various people say in the past that their process was not
stringent enough, and cite specific examples.  Also, I am aware of cases
where vendors didn't pass initially for stuff like easy DoS attacks and all
sorts of nonsense.  Since they do catch at least some problems and make the
vendors fix them, I'd put them in the category of not perfect, but helping
make things better.  Because of my knowledge of some of the things they
have caught, I am most certainly _not_ up for nothing at all.  If the
vendor can't get it past them, trust me - you really don't want to be using
it.  You probably want to apply additional criteria beyond that (which is
something where I'd be interested in seeing what various people think).

> Caveat:  I don't actually know the particulars of ICSA and the merrits of
their
> evaluation.  I HAVE been confused in the past about just exactly what they
are,
> and when I queried a booth bunny, got a waffly answer that clarified nothing.

I know a little, but don't have a definative list.  Booth bunnies often
give waffly answers...

> Crispin, fellow PhD for what it's worth :-)

8-)


David LeBlanc
dleblanc () mindspring com