Snort mailing list archives
Re: Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly
From: Anna <Anna () sonru com>
Date: Fri, 27 Oct 2017 12:34:18 +0100
HelloCynthia,
As I said before, I copied the config from 2.9.9.0 - No changes were made to snort.conf or snort.service while I
upgraded to 2.9.9.11 (2.9.9.0 IS working, while 2.9.11 is not working with the same config)
When you will go to logrotoate for snort the command is restart not reload with the snortd starting instead of my
snort.service
This is the /etc/logrotate.d/snort from server after upgrade to 2.9.9.11 without any changes made to the file (As you
can see this unchanged file is still requesting a restart in postrotate)
# /etc/logrotate.d/snort
# $Id$
/var/log/snort/alert /var/log/snort/*log /var/log/snort/*/alert /var/log/snort/*/*log {
daily
rotate 7
missingok
compress
sharedscripts
postrotate
/etc/init.d/snortd restart 1>/dev/null || true
endscript
}
I have my own custom service for systemd and without changes to logrotate snort it is killed by syslog rotation but not
restarted as the posrotate indicate snortd to start (which was happening if you can look at my previous messages - my
snort was stopped and the snort with more options was started and hung after few minutes)
My systemd service parameters
/sbin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0
The restarted process parameters
/usr/sbin/snort -A fast -b -d -D -i eth0 -u snort -g snort -c /etc/snort/snort.conf -l /var/log/snort
Thank you Anna
On 25 Oct 2017, at 15:35, Cynthia Leonard (cyleonar) <cyleonar () cisco com> wrote: Hi Anna, Good to know that your problem is resolved. here were changes to make Snort reload when certain changes to config are made and SIGHUP is issued instead of getting restarted like it used to before. Do you think there are some config changes you are making and issuing reload? Regards Cynthia From: Snort-users [mailto:snort-users-bounces () lists snort org] On Behalf Of Anna Sent: Tuesday, October 24, 2017 3:08 PM To: Carter Waxman (cwaxman) <cwaxman () cisco com> Cc: snort-users () lists snort org Subject: Re: [Snort-users] Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Ok, I solved the problem - My restart (snort[32171]: *** Caught Term-Signal) times are corresponding with daily syslogs logrotate /etc/logrotate.d/snort has diffrent restart postrotate - while there was no problem with that for some reason in 2.9.9.0 ==> I started my testing server with 2.9.9.0 just to see if the service was restarted (it was not, while the logrotate for syslogs run at 3:31am on 24th of Oct, Snort is running continuously from 23rd of Oct) The setup is the same for 2.9.9.0 snort (same snort.service, same snort.conf) [root@SYS_TEST-02 ~]# systemctl status snort ● snort.service - Snort NIDS Daemon Loaded: loaded (/usr/lib/systemd/system/snort.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2017-10-23 14:05:42 UTC; 19h ago Main PID: 26359 (snort) CGroup: /system.slice/snort.service └─26359 /sbin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 Oct 23 14:05:42 SYS_TEST-02 systemd[1]: Started Snort NIDS Daemon. Oct 23 14:05:42 SYS_TEST-02 systemd[1]: Starting Snort NIDS Daemon... 2.9.11 needs to be amended when someone is running off systemd in CentOS (do not know if any other distribution is also failing/hung) So if anybody is running systemctl start/stop/restart snort (with your own snort.service) needs to amend in the /etc/logrotate.d/snort postrotate #/etc/init.d/snortd restart 1>/dev/null || true /usr/bin/systemctl restart snort 1>/dev/null || true endscript I tested it in Monday/Tuesday and snort restarted successfully 5h ago. Can anyone tell me why this issue is present in 2.9.11 but not affecting 2.9.9.0? Thank you Anna On 17 Oct 2017, at 15:56, Anna <Anna () sonru com <mailto:Anna () sonru com>> wrote: OS: Centos 7.4 I did not compile it, I installed Snort from yum rpm - https://snort.org/downloads/snort/snort-2.9.11-1.centos7.x86_64.rpm <https://snort.org/downloads/snort/snort-2.9.11-1.centos7.x86_64.rpm> gdb really not working as cannot find the debuginfo for snort and daq pkg gdb /sbin/snort 1375 Reading symbols from /usr/sbin/snort-plain...(no debugging symbols found)...done. Attaching to program: /sbin/snort, process 1375 Reading symbols from /lib64/libnghttp2.so.14...Reading symbols from /lib64/libnghttp2.so.14...(no debugging symbols found)...done. (no debugging symbols found)...done. Loaded symbols for /lib64/libnghttp2.so.14 Reading symbols from /lib64/libdnet.1...Missing separate debuginfo for /lib64/libdnet.1 Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/cd/00c325aa44135552d31222ba244cb8f07fb761.debug Reading symbols from /lib64/libdnet.1...(no debugging symbols found)...done. (no debugging symbols found)...done. Loaded symbols for /lib64/libdnet.1 Reading symbols from /lib64/libpcre.so.1...Reading symbols from /usr/lib/debug/usr/lib64/libpcre.so.1.2.0.debug...done. done. Loaded symbols for /lib64/libpcre.so.1 Reading symbols from /lib64/libnsl.so.1...Reading symbols from /usr/lib/debug/usr/lib64/libnsl-2.17.so.debug...done. done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/libm.so.6...Reading symbols from /usr/lib/debug/usr/lib64/libm-2.17.so.debug...done. done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/libcrypto.so.10...Reading symbols from /usr/lib/debug/usr/lib64/libcrypto.so.1.0.2k.debug...done. done. Loaded symbols for /lib64/libcrypto.so.10 Reading symbols from /lib64/libdl.so.2...Reading symbols from /usr/lib/debug/usr/lib64/libdl-2.17.so.debug...done. done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libsfbpf.so.0...Reading symbols from /lib64/libsfbpf.so.0...(no debugging symbols found)...done. (no debugging symbols found)...done. Loaded symbols for /lib64/libsfbpf.so.0 Reading symbols from /lib64/libpcap.so.1...Reading symbols from /usr/lib/debug/usr/lib64/libpcap.so.1.5.3.debug...done. done. Loaded symbols for /lib64/libpcap.so.1 Reading symbols from /lib64/libz.so.1...Reading symbols from /usr/lib/debug/usr/lib64/libz.so.1.2.7.debug...done. done. Loaded symbols for /lib64/libz.so.1 Reading symbols from /lib64/libpthread.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libpthread-2.17.so.debug...done. done. [New LWP 1377] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...Reading symbols from /usr/lib/debug/usr/lib64/libc-2.17.so.debug...done. done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/usr/lib64/ld-2.17.so.debug...done. done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libnss_files.so.2...Reading symbols from /usr/lib/debug/usr/lib64/libnss_files-2.17.so.debug...done. done. Loaded symbols for /lib64/libnss_files.so.2 Reading symbols from /usr/lib64/snort-2.9.11_dynamicengine/libsf_engine.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicengine/libsf_engine.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_dce2_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_dce2_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_dnp3_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_dnp3_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_dns_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_dns_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_ftptelnet_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_ftptelnet_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_gtp_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_gtp_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_imap_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_imap_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_modbus_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_modbus_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_pop_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_pop_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_reputation_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_reputation_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_sdf_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_sdf_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_sip_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_sip_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_smtp_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_smtp_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_ssh_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_ssh_preproc.so Reading symbols from /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_ssl_preproc.so...done. Loaded symbols for /usr/lib64/snort-2.9.11_dynamicpreprocessor//libsf_ssl_preproc.so 0x00007ff26b57da3d in poll () at ../sysdeps/unix/syscall-template.S:81 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) Missing separate debuginfos, use: debuginfo-install snort-2.9.11-1.x86_64 daq-2.0.6-1.x86_64 libnghttp2-1.21.1-1.el7.x86_64 when I run debuginfo-install Could not find debuginfo for main pkg: 1:snort-2.9.11-1.x86_64 Could not find debuginfo pkg for dependency package libnghttp2-1.21.1-1.el7.x86_64 Could not find debuginfo pkg for dependency package daq-2.0.6-1.x86_64 (Is there a separate repo for debuginfo for snort?) I also notice that when the process restarts and hangs there are different parameters running (I am running systemd start with parameters /sbin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 - while the restarted process is using those /usr/sbin/snort -A fast -b -d -D -i eth0 -u snort -g snort -c /etc/snort/snort.conf -l /var/log/snort) Thank you Anna On 17 Oct 2017, at 14:43, Carter Waxman (cwaxman) <cwaxman () cisco com <mailto:cwaxman () cisco com>> wrote: Can you provide the OS and C runtime / build chain you are using? Also, can you attach gdb to the stopped process and send the backtrace? Thanks, Carter From: Snort-users <snort-users-bounces () lists snort org <mailto:snort-users-bounces () lists snort org>> on behalf of Anna <Anna () sonru com <mailto:Anna () sonru com>> Date: Tuesday, October 17, 2017 at 7:21 AM To: "snort-users () lists snort org <mailto:snort-users () lists snort org>" <snort-users () lists snort org <mailto:snort-users () lists snort org>> Subject: [Snort-users] Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Hello, I have another problem now. I upgraded the 2.9.9.0 to 2.9.11 and the problem with snort using too much memory went away. It got replaced with Snort working for few hours and terminating while the top is showing that snort is still running. Snort was upgraded and started on 13th of Oct at 9:31 am Oct 13 09:31:46 DEV_SERVER systemd: Started Snort NIDS Daemon. Oct 13 09:31:46 DEV_SERVER systemd: Starting Snort NIDS Daemon... Oct 13 09:31:56 DEV_SERVER kernel: device eth0 entered promiscuous mode then on 14th of Oct at 3:21am it stopped and tried to restart (probably hanged that is why OS is still showing snort as running) Oct 14 03:21:01 DEV_SERVER systemd: Stopping SYSV: snort is a lightweight network intrusion detection tool that currently detects more than 1100 host and network vulnerabilities, portscans, backdoors, and more.... Oct 14 03:21:02 DEV_SERVER snort: *** Caught Term-Signal Oct 14 03:21:02 DEV_SERVER kernel: device eth0 left promiscuous mode Oct 14 03:21:03 DEV_SERVER snortd: Stopping snort: [ OK ] Oct 14 03:21:03 DEV_SERVER systemd: Starting SYSV: snort is a lightweight network intrusion detection tool that currently detects more than 1100 host and network vulnerabilities, portscans, backdoors, and more.... Oct 14 03:21:03 DEV_SERVER snort[12509]: Running in IDS mode Oct 14 03:21:03 DEV_SERVER snort[12509]: Oct 14 03:21:03 DEV_SERVER snort[12509]: --== Initializing Snort ==-- Oct 14 03:21:03 DEV_SERVER snort[12509]: Initializing Output Plugins! Oct 14 03:21:03 DEV_SERVER snort[12509]: Initializing Preprocessors! Oct 14 03:21:03 DEV_SERVER snort[12509]: Initializing Plug-ins! Oct 14 03:21:03 DEV_SERVER snort[12509]: Parsing Rules file "/etc/snort/snort.conf" Oct 14 03:21:03 DEV_SERVER snort[12509]: PortVar 'HTTP_PORTS' defined : Oct 14 03:21:03 DEV_SERVER snort[12509]: [ 80:81 443 8000 8008 8080:8081 8085 8088 ] Oct 14 03:21:03 DEV_SERVER snort[12509]: Oct 14 03:21:03 DEV_SERVER snort[12509]: PortVar 'SHELLCODE_PORTS' defined : Oct 14 03:21:03 DEV_SERVER snort[12509]: [ 0:79 81:65535 ] Oct 14 03:21:03 DEV_SERVER snort[12509]: Oct 14 03:21:03 DEV_SERVER snort[12509]: PortVar 'ORACLE_PORTS' defined : Oct 14 03:21:03 DEV_SERVER snort[12509]: [ 1024:65535 ] Oct 14 03:21:03 DEV_SERVER snort[12509]: Oct 14 03:21:03 DEV_SERVER snort[12509]: PortVar 'SSH_PORTS' defined : Oct 14 03:21:03 DEV_SERVER snort[12509]: [ 22 ] Oct 14 03:21:03 DEV_SERVER snort[12509]: Oct 14 03:21:03 DEV_SERVER snort[12509]: PortVar 'FTP_PORTS' defined : Oct 14 03:21:03 DEV_SERVER snort[12509]: [ 21 2100 3535 ] Oct 14 03:21:03 DEV_SERVER snort[12509]: Oct 14 03:21:03 DEV_SERVER snort[12509]: PortVar 'SIP_PORTS' defined : Oct 14 03:21:03 DEV_SERVER snort[12509]: [ 5060:5061 5600 ] Oct 14 03:21:03 DEV_SERVER snort[12509]: Oct 14 03:21:03 DEV_SERVER snort[12509]: PortVar 'FILE_DATA_PORTS' defined : Oct 14 03:21:03 DEV_SERVER snort[12509]: [ 80:81 110 143 443 8000 8008 8080:8081 8085 8088 ] This is happening to all servers I upgraded so this is not isolated incident. How to put the snort activity logs in debug and redirect them from messages (not the u2 logs that are going to barnyard)? Also when the Snort is about the terminate those logs are created snort.log.1507951274 (there is not -l flag in the start command or any other output files in snort.conf) they contain the detected traffic logs ^@^D<8a>Óï^@PKr^W? ô<82>\<80>^X^O¦^MÉ^@^@^A^A^H ^Y)×<9e><94>ÍgHEAD http://52.125.242.20:80/PMA/ <http://52.125.242.20/PMA/> HTTP/1.1^M Connection: Keep-Alive^M Keep-Alive: 300^M User-Agent: Mozilla/5.0 Jorgee^M Host: 52.125.242.20^M Alerts log is updated with the alerts from Snort that should go to barnyard2/Snorby 10/17-11:12:09.240501 [**] [120:3:1] (http_inspect) NO CONTENT-LENGTH OR TRANSFER-ENCODING IN HTTP RESPONSE [**] [Classification: Unknown Traffic] [Priority: 3] {TCP} 10.0.4.224:80 -> 10.0.4.138:46108 But Snort is not running according to systemd since Oct 14 03:21:02 DEV_SERVER snort[32171]: *** Caught Term-Signal When I try to start snort with systemd without killing the one that is “running” according to OS, they are two processes running then. Anybody can shed any light in what is going on with my upgraded Snort? Thank you Anna _______________________________________________ Snort-users mailing list Snort-users () lists snort org <mailto:Snort-users () lists snort org> Go to this URL to change user options or unsubscribe: https://lists.snort.org/mailman/listinfo/snort-users <https://lists.snort.org/mailman/listinfo/snort-users> Please visit http://blog.snort.org <http://blog.snort.org/> to stay current on all the latest Snort news!
_______________________________________________ Snort-users mailing list Snort-users () lists snort org Go to this URL to change user options or unsubscribe: https://lists.snort.org/mailman/listinfo/snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news! Please follow these rules: https://snort.org/faq/what-is-the-mailing-list-etiquette
Current thread:
- Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Anna (Oct 17)
- Re: Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Carter Waxman (cwaxman) via Snort-users (Oct 17)
- Re: Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Anna (Oct 17)
- Re: Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Anna (Oct 24)
- Re: Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Cynthia Leonard (cyleonar) via Snort-users (Oct 25)
- Re: Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Anna (Oct 27)
- Re: Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Anna (Oct 17)
- Re: Problems with Snort 2.9.11.0 - Snort is terminating unexpectedly Carter Waxman (cwaxman) via Snort-users (Oct 17)