Re: Deployment Sizes? was: anyone trying kickfire to improve SQL performance?

[Jason <security () brvenik com>]

A very reasonable approach. Packet loss can cause a number of issues but 
low percentages shouldn't be too problematic.

Using affinity shelters one instance from issues with the other, make 
sure you handle interrupts properly too.

Things to look for with packet loss:

- Packets that seems to have mixed protocol content, if you get into 
higher percentage loss you may have to use zero_flushed_packets 
otherwise you can get buffer remnants as artifacts because of reassembly 
having gaps in the data stream

- cascading loss, packet loss can cause subsystems to work harder 
creating a cascading loss

- Errant alerts. Caused by content being left over as detailed above.

- state mismatch, loss in the setup of a session could cause mid-state 
sessions to be incorrectly identified.

Your goal should be 0 loss but it is practical to accept that some loss 
may happen during bursts and peak hours, your security posture is 
generally minimally affected because an attacker cannot reliably predict 
when the loss will occur and if it will be their packet.

SQL having its own resources is good, keep in mind that doing it 
directly from the engine is a blocking operation. Any DB work that takes 
longer then the time between packets will result in loss because the 
engine cannot process the packet. Decouple output from input. EG: use 
unified output and barnyard etc.

Memory is more important than processor in many cases, make sure you 
allocate enough memory to the snort processes to handle everything. Many 
times packet loss can be resolved simply by allocating more memory.

Stewart L wrote:
> I figured we'd add until we start dropping too many packets.   The CPU load
> on each core is only about 45% right now and we're dropping less than 1% of
> packets through the box.  We're also doing some processor affinity stuff and
> dedicating a couple cores to SQL and each instance of snort gets it's own
> core as well.
>
> I'd be interested in hearing from other folks doing large setups...
>
> Stewart
>
>
> On Sat, May 3, 2008 at 5:13 PM, Jason Haar <Jason.Haar () trimble co nz> wrote:
>
> > Stewart L wrote:
> > > Well, I wasn't in charge of the deployment. I handed it off to one of
> > > the guys on my team to do the research and recommendations.
> > >
> > > Part of the problem is that there is no SOLID advice out there on how
> > > to set up and tweak a lot of this stuff.  We have the oreilly books
> > > and have done some searches, but there is a lot of hand waving and not
> > > a lot of solid answers.
> > There are too many variables for there to be a "one size fits all"
> > answer. That's why companies like SourceFire exist - they do all that
> > background 'thinking' for you and produce a product that 'just works'.
> >
> > You should check the solution you have actually works. 6-16 100Mbs
> > Ethernet monitors on one box is probably too many. Unless you've
> > cherry-picked the motherboard,Ethernet cards, etc. And I'm assuming
> > they're 100M - if they are Gb - you almost certainly have a problem.
> >
> >
> > > So, you're saying that if I were to have another machine do the actual
> > > capture and a separate database machine, I'd be better off in the long
> > > haul?  That should be pretty easy to set up.
> > Yup - you won't get all the hard SQL work interfering with the hard
> > packet sniffing work. And barnyard of course instead of native SQL
> > support.
> >
> > --
> > Cheers
> >
> > Jason Haar
> > Information Security Manager, Trimble Navigation Ltd.
> > Phone: +64 3 9635 377 Fax: +64 3 9635 417
> > PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
> >
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> > Don't miss this year's exciting event. There's still time to save $100.
> > Use priority code J8TL2D2.
> >
> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users<https://lists.sourceforge.net/lists/listinfo/snort-usersSnort-users>list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
>
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users