Re: ACID/BASE vs PRELUDE

[Kris Karas <ktk () enterprise bidmc harvard edu>]

Gene R Gomez wrote:

> We've tested this new schema up to about 480K+ events, and Prewikka 
> can render that in about 3 seconds on decent hardware.

Hello Gene -

That's good news for those considering Prelude.  It might be nice to put 
up a feature comparison (similar for that from Aanval) showing 
differences between the open source and commercial versions.

At over 500 queries per second, we knew that the limiting factor was the 
schema, and not really the DB server, even though the latter is at 
present running on older hardware.

> Now, on another note, I did some research for the team on ways to 
> mitigate this from the database server side.  Most of the default 
> MySQL settings are pretty bad in terms of allotted RAM and cache space 
> for both queries and indices.

I think a lot of the headache could be eased by better use of unions and 
larger result sets.  The shear number of discrete queries was what was 
killing our performance.  Granted, we tune MySQL for much better 
performance; even the my-huge.cnf file needs some additional tweaking.  
On our linux/mysql logging database server (which gets about 150 
inserts/second) we also tune /proc/sys/vm/* to basically tell the OS to 
only flush dirty memory to disk once in a blue moon (bad for 
reliability, good for DB performance at those levels).

One last comment, somewhat off-topic for snort-users, and perhaps 
addressed in a newer version of prelude-manager: I could not find any 
way of getting prelude-manager to periodically retry connecting to its 
upstream peer (whether that be another prelude-manager or the DB).  So 
if I bump mysql, or restart prelude-manager on the SQL box, then I have 
to go and restart it on each snort sensor.  When the connection goes 
down, prelude-manager looks for a fail-over server to transfer to; if 
none is available, it just gives up; there doesn't appear to be any way 
to get it to queue data and retry every minute or so.  Or maybe I'm just 
being a pinhead and missed the obvious.  :-)

Kris


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users