RE: Problem with MYSQL/ACID And Large Database

["Pacheco, Michael F." <MPacheco () elcom com>]

Jeff,
 
Unless you've got a healthy server running that db, 100,000 entries will
slow you down considerably.  Setup a 2nd server with MySQL, import the
schema (create_mysql.sql) for snort, set your passwords and db permissions
and go to the ACID box - I believe the file is acid_conf.php - the setup for
the archive instance is right under the setup for the primary instance.
Archive off as much as you can, get the primary down to 20-30k ( I keep mine
around 15k) - work the alerts you get, if you can't or want to save them for
data mining archive them and set up alert groups on the archive site.  I run
a whole separate instance of ACID just looking at my archive site so it will
not performance degrade my primary. You can to this chain style as many
times as you want - one small box holding 30 day data, another one 60 day -
etc... as many as you have room and boxes for.  I run my back end archive
db's on converted Compaq 600 desktops with extra memory and HD with separate
instances of ACID - modify the header page title so I know what db I'm
looking at, add an index page for the group to click through my primary site
into the archives and you've got yourself a homegrown distributed setup -
throw in Barnyard on the sensors and snortsnarf on the archives and your all
set!  The pig can squeal very nicely if you plan it out and stay on top of
him!!
 
IMHO - your mileage might vary. (Sorry for the soap box, I get cranked up
sometimes - my beanie propeller is spinning to fast now)
Mike
 
Michael F. Pacheco  CCNA, MCSE
Network Analyst
Elcom International
10 Oceana Way
Norwood, Ma. 02062
Direct   781-501-4258
Fax      781-762-1540
mpacheco () elcom com
 
-----Original Message-----
From: Maynard, Jeff S. [mailto:Jeff.Maynard () banctec com] 
Sent: Monday, March 03, 2003 10:32 AM
To: 'Pacheco, Michael F.'; Maynard, Jeff S.;
'snort-users () lists sourceforge net'
Subject: RE: [Snort-users] Problem with MYSQL/ACID And Large Database
 
I am currently running around 100,000 events in the acid_event file.  Are
there some notes on how to set up archieving?
-----Original Message-----
From: Pacheco, Michael F. [mailto:MPacheco () elcom com] 
Sent: Monday, March 03, 2003 9:29 AM
To: 'Maynard, Jeff S.'; 'snort-users () lists sourceforge net'
Subject: RE: [Snort-users] Problem with MYSQL/ACID And Large Database
Had the same problem, its DNS related, I fixed it 2 ways.   Got rid of IE
and went to Netscape (Mozilla on RH 8.0 works well also) or if you need IE
for some reason, put a hosts file entry on your workstation pointing at the
Acid site.  I did the hosts entry first and performance over IE picked up
dramatically,  installed Netscape 7.0 on the workstation and ACID
performance was much better - IE is still liveable, but Netscape just seems
to handle php code better.
 
Of course this is workstation related, if your carrying 30k plus alerts in
your MySQL db instance then you really need to set up an archive instance
off the primary db server - but that's a different story.
 
Hope that helps,
 
Mike
 
Michael F. Pacheco  CCNA, MCSE
Network Analyst
Elcom International
10 Oceana Way
Norwood, Ma. 02062
Direct   781-501-4258
Fax      781-762-1540
mpacheco () elcom com
 
-----Original Message-----
From: Maynard, Jeff S. [mailto:Jeff.Maynard () banctec com] 
Sent: Monday, March 03, 2003 9:53 AM
To: 'snort-users () lists sourceforge net'
Subject: [Snort-users] Problem with MYSQL/ACID And Large Database
 
I am having a problem with the ACID running against the MYSQL database.
This is a new installation of Snort and I am still working on tuning the
false positives so there is a tremendous amount of data in the database.
The problem that I am running into is that I cannot get the ACID console to
load in any reasonable timeframe which results in a timeout of the browser.
I have increased my PHP timeout settings which helps for a day or so but
again the database grows and it starts to time out again.  I end up having
to go into the acid_events database and manually delete records which I
would prefer not to do until I have had a chance to review them in
coorelation to the rest of the data.  Has anyone else had this problem and
if so how did you correct it?