Snort mailing list archives
Re: Re: ACID/BASE vs PRELUDE
From: Gene R Gomez <gene () gomezbrothers com>
Date: Mon, 19 Sep 2005 16:08:03 -0700
Hey Kris,Yeah, this has been a big issue for many people using the Open Source version of the software; enough that we have put some time into developing a new schema that's been deployed into the commercial version of the software. We've tested this new schema up to about 480K+ events, and Prewikka can render that in about 3 seconds on decent hardware. We're currently working on back-porting portions of the schema update functionality into the Open Source version. Unfortunately, as a team we set a deadline for the 0.9 release (it's been in development for a very long time) that didn't allow for the inclusion of a schema update for the Open Source version. Now, on another note, I did some research for the team on ways to mitigate this from the database server side. Most of the default MySQL settings are pretty bad in terms of allotted RAM and cache space for both queries and indices. I'll admit, the performance is still not so good after tweaking MySQL up, but it's far better than the numbers you give below. At any rate, improvements are on their way. We can only work so fast, though. ;)
Gene R Gomez Kris Karas wrote:
We set up Prelude to funnel the output from several Snort sensors into a backend DB (MySQL) from which we used Prewikka (the python-based display engine) for viewing. Prewikka is very nice in concept, filters work well, does a decent job of organizing data visually using color-coding. The Prelude-Manager is, I think, the best part of Prelude; it allows data from various sensors to be aggregated, stored, and sent upstream to either another aggregator or else into a database for display. (This is nearly identical to the "event collector" that we used in our ISS RealSecure deployment.)Back to Prewikka for a moment... I'd prefer an interface that shows aggregate counts in descending order of severity, with an ability to drill-down by src/dst network/address and, eventually, down to individual discrete events. Prewikka instead shows all events in the order in which they were received, but allows you to apply filters if you have some clue of what you're looking for.On the negative side, Prelude-Manager's DB has serious performance issues that need to be addressed. The fundamental limiting factor is its schema, which puts most forms of data into a single table, along with a tag that identifies the nature of the item. Just pulling up one screenful of log data (50 signatures) could take several minutes. Profiling to see where the bottleneck was shows MySQL fielding some 500+ queries per second to satisfy just that one page view. If you pull up a thousand entries, for example, you might as well go out for lunch.
------------------------------------------------------- SF.Net email is sponsored by: Tame your development challenges with Apache's Geronimo App Server. Download it for free - -and be entered to win a 42" plasma tv or your very own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users
Current thread:
- ACID/BASE vs PRELUDE ddodge (Sep 19)
- Re: ACID/BASE vs PRELUDE Joel Esler (Sep 19)
- Re: ACID/BASE vs PRELUDE Kris Karas (Sep 19)
- Re: Re: ACID/BASE vs PRELUDE Gene R Gomez (Sep 19)
- Re: ACID/BASE vs PRELUDE Kris Karas (Sep 20)
- Re: Re: ACID/BASE vs PRELUDE Gene R Gomez (Sep 19)
- <Possible follow-ups>
- Re: ACID/BASE vs PRELUDE Gene R Gomez (Sep 19)
- RE: ACID/BASE vs PRELUDE Ron Jenkins (Sep 19)