Snort mailing list archives
Barnyard2 fatal error duplicate references, but there are no duplicates
From: elof () sentor se
Date: Thu, 1 Nov 2012 12:57:34 +0100 (CET)
I just upgraded to barnyard2.1.10, and it complains... I dropped and created a brand new database so that there are no old garbage data. I pre-populated the reference-system with all the metadata like I've done for years. I ran barnyard2 in testmode, but it bails: barnyard2 in self-test mode: barnyard2 -T -v -c barnyard2.conf -d /log -f snort.unified2 --pid-path /var/run Found pid path directive (/var/run) Running in Test mode --== Initializing Barnyard2 ==-- Initializing Input Plugins! Initializing Output Plugins! Parsing config file "barnyard2.conf" Found pid path directive (/var/run) WARNING database: Defaulting Reconnect/Transaction Error limit to 10 WARNING database: Defaulting Reconnect sleep time to 5 second Checking PID path... PID path stat checked out ok, PID path set to /var/run Writing PID "45577" to file "/var/run/barnyard2_mon0.pid" Chroot directory = /var/log/snort ERROR database: Query [SELECT ref_id FROM reference WHERE ref_system_id = '7' AND ref_tag = 'blog.fireeye.com/research/2010/10/feodosoff-a-new-botnet-on-the-rise.html#more';] returned more than one result [SystemCacheSynchronize()], Call to ReferencePopulateDatabase() failed [CacheSynchronize()]:, SystemCacheSyncronize() call failed. ERROR: database [DatabaseInitFinalize()]: CacheSynchronize() call failed ... Fatal Error, Quitting.. Done. Cleaning up. So I guess the "returned more than one result" is the actual problem, and the following failed cache errors are just a result of it. Now, the strange thing is that there are no multiple results for this query!
From the same machine, I run 'psql', using the same configuration as
barnyard2 use, and run the exact command as listed in the errorlog: psql -h 10.10.10.10 foo foo Password for user foo: foo=> SELECT ref_id FROM reference WHERE ref_system_id = '7' AND ref_tag = 'blog.fireeye.com/research/2010/10/feodosoff-a-new-botnet-on-the-rise.html#more'; ref_id -------- 2872 (1 row) There's just one result, so why do barnyard2 complain about "returned more than one result" ? I guess the error message is related to some other query, not the one logged on screen. So, apparently this specific url-based reference was _not_ duplicated. However, my old system that pre-populates the reference-system populates things just like they did a long time ago, before barnyard even existed and the database plugin was builtin in snort. I.e. it populates the same reference multiple times if several rules use the same reference: Example: The ref_id 7 = 'url'. This table unique. However, the following three rules would generate three rows in the reference table: alert udp $HOME_NET any -> any 53 (msg:"blah1"; ... reference:url,www.f-secure.com/weblog/archives/00002227.html; sid:2013481; rev:1;) alert udp $HOME_NET any -> any 53 (msg:"blah2"; ... reference:url,www.f-secure.com/weblog/archives/00002227.html; sid:2013482; rev:2;) alert udp $HOME_NET any -> any 53 (msg:"blah3"; ... reference:url,www.f-secure.com/weblog/archives/00002227.html; sid:2013483; rev:1;) ref_id ref_system_id ref_tag 17 7 www.f-secure.com/weblog/archives/00002227.html 18 7 www.f-secure.com/weblog/archives/00002227.html 19 7 www.f-secure.com/weblog/archives/00002227.html I'm not sure, but I can guess that the new barnyard2, in some other query that is not logged, complain about these kind of duplicates. 1) Should there only be only ONE 'www.f-secure.com/weblog/archives/00002227.html' in the database, and the signatures 2013481, 2013482 and 2013483 should all reuse it, not having their own ref_id instance? 2) In the new barnyard2.1.10, you seem to have added some kind of pre-population of the reference system. Does this mean that there's no longer any need to pre-populate it using a separate system? If so that would be great - one system less to deal with. Also, if I'm correct in my assumptions above, the problem should not appear at all. /Elof my barnyard2.conf: var SENSOR_NAME foo var DB_USER foo var DB_PASSWORD foo var INTERFACE mon0 var DB_HOST 10.10.10.10 var DB_NAME foo config reference_file: /usr/local/etc/snort/reference.config config classification_file: /usr/local/etc/snort/classification.config config gen_file: /usr/local/etc/snort/gen-msg.map config sid_file: /usr/local/etc/snort/barnyard2.sid-msg.map config logdir: /var/log/snort config hostname: $SENSOR_NAME config interface: $INTERFACE config alert_with_interface_name config chroot: /var/log/snort config set_gid: snort config set_uid: snort config show_year config umask: 022 input unified2 output log_tcpdump: $SENSOR_NAME-barnyard2.tcpdump output database: log, postgresql, user=$DB_USER password=$DB_PASSWORD dbname=$DB_NAME host=$DB_HOST sensor_name=$SENSOR_NAME Version: ______ -*> Barnyard2 <*- / ,,_ \ Version 2.1.10 (Build 310) |o" )~| By Ian Firns (SecurixLive): http://www.securixlive.com/ + '''' + (C) Copyright 2008-2012 Ian Firns <firnsy () securixlive com> ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct _______________________________________________ Snort-devel mailing list Snort-devel () lists sourceforge net https://lists.sourceforge.net/lists/listinfo/snort-devel Archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel Please visit http://blog.snort.org for the latest news about Snort!
Current thread:
- Barnyard2 fatal error duplicate references, but there are no duplicates elof (Nov 01)