Re: Penetration testing via shrinkware

["Paul D. Robertson" <proberts () clark net>]

On Sun, 20 Sep 1998, John McDermott wrote:

> > HTTP is an open-ended protocol specification with some _limitless_ size 
> > specifications, I submit that it is beyond "difficult" to verify correct 
> > functionality of a layer 5 transport protocol.  Testing just buffer 
> > overflows on limitless length objects would seem to be less than an ideal 
> > situation. 
>
> Absolutly.

Drives me to drink too ;)

> >      Proxies are much easier to verify than stateful filters under 
>
> No doubt about that.

I wonder if it's all that obvious to folks though.  I've never seen it as 
a part of an asessment metric for any "public" evaluation criteria.  That 
tends to bother me, it seems that with most "tests" more time is spent on 
how cute the interface is than on the principles of design of the engine.

> > the same circumstances, but once again, the source code is probably going 
> > to give you a much higher level of assurance that oversized objects are 
> > correctly handled unless you don't go look at the souce to the library 
> > routines as well, in which case you can either do that, or accept a lower 
> > level of assurance by banging against the calls with a substantial set of 
> > test data.
>
>
> I do not disagree with this. My real concern is that you have to know what 
> to look for.  If the designers of the versions of the code which have 
> "security holes" had known what to look for, they would have (hopefully!) 
> done things correctly.  My real concern is that the inspectors have to know 
> what to look for.

Absolutely!  However, it is _possible_ that if folks gave enough of a 
hoot, a good deal of this could be driven by a parser and tools that 
"best common practice" could be shifted to some sort of code audit with 
Inspector Clues[eu] (ugh, sorry).  I'm all for giving those old TCB labs 
commercial business, lots of it too.

> You are also making an assumption that I was not: that the tester has 
> access to the source code.  I doubt if I could go to vendor "X" and tell 
> them that I want to verify the security of a firewall for my client and 
> could I have a peek at the source.   Maybe if my client were big enough I 
> could, but for many of us that is not an option.  Just out of curiousity 
> does ICSA look at the source for certification?

But it *should* be an option.  With the appropriate NDA in place, _why_ 
would a vendor have a problem with a 3rd party verifying their code?  
Once again, we, as implementors, consultants, customers, and the industry 
*need* to bring out the bar so we can raise the darned thing.  We've got 
freeware authors who religiously run everything through products like 
Purify *and* hand out source code, yet there's no outcry when commercial 
"black box" companies produce code which fails the buffer overrun test in 
the first 90 days of real-world usage.  Shouldn't those vendors be held 
to an even higher standard?

The last vendors I've asked about source code was happy to have me look 
under NDA (not that I'm the world's best code auditor - in fact I pretty much 
suck at it).  Maybe if we all started asking to see the code, it'd be a 
norm, and not some monumentous occasion that takes 8 lawyers, a 
Presidential cigar and a parrot to institute.   

Ok, this is *obviously* a soapbox issue for me, but big or small (and 
it's easy for me to say - my employer is large) we should start raising 
the bar, or at least trying to.  

My assumptions are based on the "If the vendor won't play nicely with me, 
I find a new vendor" model.  Most of the time it works because it tends 
to be in their interests to play well too.  Sometimes it doesn't, then I 
have to make a call on staying with the vendor or moving on.  

You probably assume "not worth asking", where I assume "I get to see what I 
want so long as we sign the right paperwork".  It works for me about 8 times 
out of 10, perhaps it's worth a try next time?  Heck, it's to the 
vendor's advantage if you find something wrong.  

Paul
-----------------------------------------------------------------------------
Paul D. Robertson      "My statements in this message are personal opinions
proberts () clark net      which may have no basis whatsoever in fact."
                                                                     PSB#9280