Re: Cisco LEAP

[johnadams <johnadams () apple com>]

On Monday, November 3, 2003, at 11:59 AM, Rob Shein wrote:

> It's not a question of peak performance as much as consistency.  Flat 
> files
> aren't meant to work this way; that's largely why database 
> applications work
> the way they do in the first place.  If something like paging competes 
> for
> drive access just long enough, the whole thing can go to hell.  When 
> you're
> opening a graphic or text file completely into memory to view or edit 
> it?
> For that, sure, a flat file is faster.  But when you're streaming 
> through a
> flat file that's dozens of gigs in size, over an extended period of 
> time
> while running the data into a memory and processor-intensive program 
> at the
> same time?  Try it, and just see how quickly that works over the 
> length of
> the entire file compared to a database :)

The real issue here is the right tool for the job -- we're talking 
about a file with many passwords in it, which ostensibly would be under 
a few megabytes in size. You could mmap() the entire thing into memory 
and get consistent access without the use of a database. Memory is 
cheap these days.

One thing that I see much of in software design is an overwhelming 
desire to put everything into a database with complete disregard for 
performance,

I used to work at Inktomi, and we used very little in the way of 
databases to hold massive datasets (all web pages on the Internet.) We 
avoided databases for performance reasons, and saw serious gains 
because of customized code that read flat files filled with structures.

I guess the thing to remember here is that eventually the database has 
to write your data out to disk, and when that happens, it'll be placed 
on the disk in a file, using an fwrite() and a modicum of indexes into 
the data. Even programs like mysql eventually write their data out as 
BerkeleyDB files.

-john
(posting far outside the scope of pen-test now)


---------------------------------------------------------------------------
Network with over 10,000 of the brightest minds in information security
at the largest, most highly-anticipated industry event of the year.
Don't miss RSA Conference 2004! Choose from over 200 class sessions and
see demos from more than 250 industry vendors. If your job touches
security, you need to be here. Learn more or register at
http://www.securityfocus.com/sponsor/RSA_pen-test_031023
and use priority code SF4.
----------------------------------------------------------------------------