Re: Inline snort negative impact on network

[Charlie Heselton <charles.heselton () gmail com>]

Oh, daq configuration from snort.conf:
config daq: afpacket
config daq_dir: /usr/lib64/daq
#config daq_mode: passive
config daq_mode: inline
config daq_var: buffer_size_mb=1024

On Thu, Nov 13, 2014 at 4:41 PM, Charlie Heselton <
charles.heselton () gmail com> wrote:

> On Thu, Nov 13, 2014 at 2:57 PM, Y M <snort () outlook com> wrote:
>
> > Date: Thu, 13 Nov 2014 12:09:45 -0800
> > From: charles.heselton () gmail com
> > To: snort-users () lists sourceforge net
> > Subject: Re: [Snort-users] Inline snort negative impact on network
> >
> > YM,
> >
> > I'm not sure what "lro, gro, and the rest of the gang" means, or what is
> > involved in dis-/enabling them.
> >
> > I have tweaked the RX/TX buffers.  Here are (some of) the tuning changes
> > I've made in /etc/sysctl.conf:
> > # Performance settings
> > net.core.netdev_max_backlog = 10000
> > net.core.r mem_default = 16777216
> > net.core.rmem_max = 33554432
> > net.ipv4.tcp_mem = 194688 259584 389376
> > net.ipv4.tcp_rmem = 1048576 4194304 33554432
> > net.ipv4.tcp_no_metrics_save = 1
> > net.ipv4.tcp_sack = 0
> > # IF also in Inline mode:
> > net.core.wmem_default = 16777216
> > net.core.wmem_max = 33554432
> > net.ipv4.tcp_wmem = 1048576 4194304 16777216
> > # Memory handling ? not that important
> > vm.overcommit_memory=2
> > vm.overcommit_ratio = 50
> >
> > These tunings are based on various article I've found while googling.
> >
> > I will tak a look at the http_inspect configuration.
> >
> > Thanks again, for the advice.
> >
> > ## Sorry I wasn't clear. These are the NIC offloading options which are
> > not desired when sniffing packets as they "manipulate" how packets are
> > presented to kernel/Snort.  For example, for LRO and GRO:
> > http://manual.snort.org/node7.html. There are other offloading features
> > that may need to be disable as well, such as GSO, TSO. Run ethtool -k
> > <interface> to see what is enabled/disabled and then use ethtool -K to
> > disable them as mentioned in the link.
> >
> > What I meant by the RX/TX buffers are the NIC ones, not only the
> > kernel's. Use the ethtool again (with -g and -G) to determine/modify the
> > values of the buffers: http://linux.die.net/man/8/ethtool. What daq mode
> > are running?
> >
> > YM
> >
> > This is what's on by default, on my system:
> ethtool -k enp2s0 | grep "on$"
> rx-checksumming: on
> generic-receive-offload: on
> rx-vlan-offload: on
> tx-vlan-offload: on
>
> I'm assuming GRO is generic-receive-offload?  I'll play around with
> disabling the others.  No VLANs in my setup.
>
> ethtool -g enp1s0 just gives me an error:
> Ring parameters for enp1s0:
> Cannot get device ring settings: Operation not supported
>
> Did I miss something in the kernel config?
>
> I did bump the txqueuelen, with ifconfig, from 1000 to 10000 (based on one
> article I found).  Another article I read said that all interfaces involved
> needed to be in promisc mode.  That is now also set for the 2 bridge
> interfaces, and the connected interface on the linux firewall.  I can't do
> anything with the dumb-switch being used on the other end.
>
> Hopefully I will get a chance to do some more testing tonight, with all of
> these tweaks in place.
>
> Thanks.
> -Charlie
>
> > On Thu, Nov 13, 2014 at 10:07 AM, Y M <snort () outlook com> wrote:
> >
> > Date: Thu, 13 Nov 2014 09:46:24 -0800
> > Subject: Re: [Snort-users] Inline snort negative impact on network
> > From: charles.heselton () gmail com
> > To: snort () outlook com
> > CC: snort-users () lists sourceforge net
> >
> >
> >
> > On Wed, Nov 12, 2014 at 10:59 PM, Y M <snort () outlook com> wrote:
> >
> > I would say tuning; NIC (gro, lro, etc), kernel (networking stack), and
> > Snort itself (number of rules/processors, etc). Since you are already on
> > Snort 2.9.7.0, why not using daq 2.0.4? And there is the
> > "unknown/unexpected" hardware behavior. If all the tuning does not improve
> > things, see if you can test with different NICs if possible.
> >
> > YM
> >
> >
> >
> > I've done some sysctl tuning, but it hasn't seemed to make much of a
> > difference.  ifconfig shows that there are 5 (out of 600K+) dropped RX
> > packets on only 1 of the 2 bridged interfaces.  All of the other
> > error-indicating counters are 0.  Again, system resources remain low when
> > the system is inline.  So I don't know that performance is really an issue.
> >
> > Using daq 2.0.2 because that's what's avilable in Gentoo's software
> > repository.  If/when 2.0.4 becomes available, I'll upgrade and see if it
> > makes a difference.
> >
> > I suspect that snort is dropping random packets, but have no way to
> > confirm.
> >
> > Thanks for the response YM, Still hoping for some useful advice from the
> > community.
> >
> >
> > # I see. Have you also disabled lro, gro, and the rest of the gang? They
> > have been the most part of the issue when setting up Snort inline. And
> > while you are at the NIC level, you may also want to adjust RX/TX buffers.
> >
> > Another thing that I would tune in specific is the http_inspect
> > preprocessor, and then move to the remaining configurations, like disabling
> > unwanted preprocessors and rules..Hope this can helps.
> >
> > YM
> >
> >
> >
> > ------------------------------
> > Date: Wed, 12 Nov 2014 20:31:31 -0800
> > From: charles.heselton () gmail com
> > To: snort-users () lists sourceforge net
> > Subject: [Snort-users] Inline snort negative impact on network
> >
> >
> > I'm attempting to install/configure a standalone, inline snort box.  When
> > I have the sensor inline, with snort running, the traffic seems to be
> > flowing properly; snort is alerting, as expected.
> >
> > However, browsing the web, and downloads, becomes significantly impacted.
> >  speedtest.net fails to load.  wget downloads files at ~6Kbps, when it
> > should be closer to 6Mbps.
> >
> > The question is why?
> >
> > Hardware:  Intel Celeron 4 core, 8GB RAM, 64GB SSD, dual Gigabit
> > (Realtek) NICs onboard, USB3.0->Gigabit dongle NIC (for admin).
> >
> > Software:  Gentoo x86_64 linux; kernel 3.16.5; snort 2.7.0; daq 2.0.2.
> >
> > When snort is running, and traffic is passing, both gkrellm and top show
> > almost 0 CPU activity.  This is on a relatively low traffic, home network,
> > so I wouldn't expect the system to be loaded.  The admin interface shows
> > more activity than the 2 bridged interfaces.
> >
> > What gives?  Any advice appreciated.
> >
> > Thanks,
> > Charlie
> >
> >
> >
> > ------------------------------------------------------------------------------
> > Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for
> > $9/Month. Get alerted through email, SMS, voice calls or mobile push
> > notifications. Take corrective actions from your mobile device.
> > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
> > _______________________________________________ Snort-users mailing list
> > Snort-users () lists sourceforge net Go to this URL to change user options
> > or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users
> > <https://lists.sourceforge.net/lists/listinfo/snort-usersSnort-users>
> > list archive:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> > Please visit http://blog.snort.org to stay current on all the latest
> > Snort news!
> >
> >
> >
> >
> > ------------------------------------------------------------------------------
> > Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for
> > $9/Month. Get alerted through email, SMS, voice calls or mobile push
> > notifications. Take corrective actions from your mobile device.
> > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
> > _______________________________________________ Snort-users mailing list
> > Snort-users () lists sourceforge net Go to this URL to change user options
> > or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users
> > <https://lists.sourceforge.net/lists/listinfo/snort-usersSnort-users>
> > list archive:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> > Please visit http://blog.snort.org to stay current on all the latest
> > Snort news!
------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!