Re: Diagnosing MySQL server has gone away messages

[bleh <mailbox.size.limit () gmail com>]

See comments inline.

On 8/21/07, Jason <security () brvenik com> wrote:
> bleh wrote:
> > This would seem to be dependent on a lot of factors.
> >
> > database event rate
> > snort alert rate
> > cpu/memory/io
> > system load
> > network kbps
> > type of traffic (protocols etc.)
> > enabled rules / quality of rules
> > enabled preprocessors
> > snort variable assignment
>
> I think you are getting a bit pedantic. You are not concerned about "a
> user" you are concerned about the environment surrounding the sensor.
> Why would you needlessly add another risk factor to performance when
> there are clear solutions to it?


The job of any systems, network or ids administrator is to look at the
systems and network as a whole. Network and systems profiling is far from
being pedantic, this is called performing your job.


Do not let perfection be the enemy of good.
> > A user can do a lot more damage to a sensor with any combination of the
> > above items. For example I can make Snort stop inspecting any traffic
> > for quite a while with a single (badly written) pcre rule. There are a
> > number of preprocesors that can turn a sensor into a brick under the
> > right conditions as well. Does this mean all users should stop using
> > pcre and the smtp preprocessor? I do not believe so.
>
> Do you have something in the preprocessors you care to share with the
> development team so it can be addressed or is this conjecture?


Portscan, SMTP, DNS and BO just to name a few.


Just because you can write a poor rule does not invalidate that you
> should not do direct writes to the database. As always, there must be a
> car analogy.


I agree, a car analogy does work. Sure, I know that having on the A/C has
the potential to impact performance. However, there are still many key
factors involved such as the engine size, transmition, fuel and weight.
These factors are very similar in nature to those I addressed above. You
telling me I should *always* run with my A/C off is a blanket statement and
has very little bearing when you have no idea what kind of car I drive and
how I drive it.


Just because you can go faster than conditions warrant does not mean you
> should.
>
> > As with anything else on a sensor you need evaluate your situation and
> > set up the system accordingly. There is no one size fits all, and this
> > (IMHO) includes output plugins. For example, I have two sensors in my
> > lab (for validating new config options, new snort versions etc.)
> > watching the same traffic one with unified and one with db. I
> > continually receive the same number of alerts from both systems.
>
> I humbly assert that you are incorrect in every sense of the word. There
> is absolutely no practical advantage to using direct DB writes from the
> engine compared to using unified output and there are certainly clear
> disadvantages to using them.


My statement states there is not a "one size fits all" solution. You stating
that I am "incorrect in every sense of the word"  asserts that there is a
"one size fits all" solution. Therefore, please tell me which config,
preprocessors, rules, hardware and OS everyone in the Snort community should
be running.


-bleh

> > -bleh
> >
> > On 8/21/07, *Joel Esler* <joel.esler () sourcefire com
> > <mailto:joel.esler () sourcefire com>> wrote:
> >
> >     When snort logs to db, it takes resources away from inspecting
> >     traffic.  Missing packets.
> >
> >     --
> >     Joel Esler
> >     Sent from the road.
> >
> >     On Aug 21, 2007, at 10:33 AM, bleh <mailbox.size.limit () gmail com
> >     <mailto:mailbox.size.limit () gmail com>> wrote:
> >
> > >     Can you explain what you mean by Snort "has to stop being an IDS"?
> > >     If Snort is no longer an IDS when logging directly to a DB what is
> it?
> > >     > In order for Snort to do an insert, it has to stop being an IDS.
> > >     > Personally, I don't want my IDS to miss any packets, regardless
> > >     of the
> > >     > extremely short amount of time it takes to do a DB insert.
> > >      (Unless you
> > >     > have a very busy DB).
> > >     >
> > >     > Still would rather you use Barnyard and Unified.
> > >     >
> > >     > Joel
> > >     >
> > >     > Jason Haar wrote:
> > >     >> Joel Esler wrote:
> > >     >>> As a general recommendation for everyone on the list, Snort
> > >     should never be logging directly to the DB.
> > >     >>>
> > >     >>>
> > >     >> Can you tell me why that is? I mean, surely directly accessing a
> DB
> > >     >> doesn't introduce much latency/load *unless* it continually
> > >     generates
> > >     >> events?
> > >     >>
> > >     >> i.e. if a particular installation of snort only generates 1
> > >     event per
> > >     >> minute, is there really any difference between snort->mysql and
> > >     >> snort->barnyard?
> > >     >>
> > >     >> I could understand it being a problem if snort is having to
> > >     queue up
> > >     >> events while INSERTs are going on, but if that are spaced many
> > >     seconds
> > >     >> apart...?
> > >     >>
> > >     >
> > >     > --
> > >     >
> > >     >
> > >     > --
> > >     > joel esler | security consultant | Sourcefire | pgp is public
> > >     <http://www.geocrawler.com/redir-sf.php3?list=snort-users>
> -------------------------------------------------------------------------
> > >     This SF.net email is sponsored by: Splunk Inc.
> > >     Still grepping through log files to find problems?  Stop.
> > >     Now Search log events and configuration files using AJAX and a
> > >     browser.
> > >     Download your FREE copy of Splunk now >>
> > >     <http://get.splunk.com/>http://get.splunk.com/
> > >     _______________________________________________
> > >     Snort-users mailing list
> > >     Snort-users () lists sourceforge net
> > >     <mailto:Snort-users () lists sourceforge net>
> > >     Go to this URL to change user options or unsubscribe:
> > >     https://lists.sourceforge.net/lists/listinfo/snort-users
> > >     <https://lists.sourceforge.net/lists/listinfo/snort-users>
> > >     Snort-users list archive:
> > >     http://www.geocrawler.com/redir-sf.php3?list=snort-users
> > >     <http://www.geocrawler.com/redir-sf.php3?list=snort-users>
> >
> >
> >
> > ------------------------------------------------------------------------
> -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems?  Stop.
> > Now Search log events and configuration files using AJAX and a browser.
> > Download your FREE copy of Splunk now >>  http://get.splunk.com/
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users