RE: Question on database for Snort

["Kenneth G. Arnold" <bkarnold () cbu edu>]

I am currently working on a web-based interface also.  Our mysql database 
currently has about 630,000 alerts in it. The main page of my system 
displays the current day's activity of about 4000 alerts grouped by alert 
in descending order with a count for each alert in about 1 second. The data 
is extracted from the default snort tables with no additional indexes.  A 
similar query in ACID takes 2 1/2 minutes once you have reached the main page.

One main difference between the two is that ACID must be looking at each 
individual alert because it displays the first and last occurrences for 
each alert type. This means that the application is doing a large portion 
of the work and the database is simply supplying it with information.  I 
have my page generated from a single sql statement that returns only the 
information I need to the program.  The database does most of the work and 
the application simply displays it.

Of course the beauty of an application that you have written yourself is 
that you can make it very efficient.  I am writing mine for Solaris and 
mysql using C.  It is unknown whether the finished application will even 
run on other configurations.  ACID is designed to run on a lot more 
platforms using multiple databases and that slows it down.

Ken

At 03:32 PM 3/31/03 -0600, Paul Schmehl wrote:
> On Mon, 2003-03-31 at 13:35, Kreimendahl, Chad J wrote:
> > The majority of setups I've seen with mySQL tend to bog down greatly
> > when you approach 100k events in the db.  Postgres seems to handle much
> > more, but still has its problems as the numbers increase.  Oracle has
> > been the most stable, for those I've had experience with.  I've seen
> > several Oracle setups storing snort information running much more
> > complex front-ends than ACID... that easily store and retrieve tens of
> > millions of records without much more delay than it would a few
> > thousand.
> >
> I have never tested PostgreSQL, so I can't speak to that, but I *can*
> address one of your points above.  We are presently querying a mysql
> database with 8 million alerts in it, using a web-based interface that
> we are designing, and we are getting response times of under 3 seconds.
>
> I think the response time of any front end to a database has a lot more
> to do with how the queries are constructed than a lot of people
> realize.  For example, a similar query using ACID takes about 680
> seconds on a database with 1.5 million alerts in it.
>
> --
> Paul Schmehl (pauls () utdallas edu)
> Adjunct Information Security Officer
> The University of Texas at Dallas
> http://www.utdallas.edu/~pauls/
> AVIEN Founding Member
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: ValueWeb:
> Dedicated Hosting for just $79/mo with 500 GB of bandwidth!
> No other company gives more support or power for your dedicated server
> http://click.atdmt.com/AFF/go/sdnxxaff00300020aff/direct/01/
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users


Brother Kenneth Arnold
System Administrator
Information Technology Services
Christian Brothers University
(901) 321-4333



-------------------------------------------------------
This SF.net email is sponsored by: ValueWeb: 
Dedicated Hosting for just $79/mo with 500 GB of bandwidth! 
No other company gives more support or power for your dedicated server
http://click.atdmt.com/AFF/go/sdnxxaff00300020aff/direct/01/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users