Snort mailing list archives
CID duplication issues with ACID, snort and multiple sensors
From: W Fenwick <wfenwick () entrenet com>
Date: Fri, 22 Mar 2002 11:51:34 -0500
Hi List, We have spent the better part of a day doing in-depth debugging on this problem and have had to resort to an email to the list.We have a snort/ACID combination which has been running for several months without much of a hitch. Recently we added a second sensor with Snort 1.8.3 and it began logging to the central Postgres database also. Everything has been working well, until yesterday. First of all, we upgraded ADODB to v1.72 on Tuesday from ADODB 1.20. The fellow who usually does the operational monitoring was away Wednesday, so I took over for the day and noticed that archival was partially failing. Added 59 alert(s) to the Alert cache Ignored 414 duplicate alert(s) Successful ARCHIVE-move - 84 alert(s) This seemed odd. I wasnt sure what was going on. I later started a new session and recieved this error on pointing at the home URL for my ACID install. It was complaining sensor #5: event.cid = 2391, acid_event.cid = 0 Database ERROR:Database ERROR:ERROR: Cannot insert a duplicate key into unique index acid_event_pkey INSERT INTO acid_event (sid,cid,signature,timestamp, ip_src,ip_dst,ip_proto, sig_name, sig_priority, sig_class_id) SELECT event.sid as sid, event.cid as cid, signature, timestamp, ip_src, ip_dst, ip_proto, sig_name, sig_priority, sig_class_id FROM event INNER JOIN signature ON (signature = signature.sig_id) LEFT JOIN iphdr ON (event.sid=iphdr.sid AND event.cid=iphdr.cid) WHERE (sig_name LIKE 'spp_%') AND (event.sid = 5 AND event.cid > 0) INSERT INTO acid_event (sid,cid,signature,timestamp, ip_src,ip_dst,ip_proto, sig_name, sig_priority, sig_class_id) SELECT event.sid as sid, event.cid as cid, signature, timestamp, ip_src, ip_dst, ip_proto, sig_name, sig_priority, sig_class_id FROM event INNER JOIN signature ON (signature = signature.sig_id) LEFT JOIN iphdr ON (event.sid=iphdr.sid AND event.cid=iphdr.cid) WHERE (sig_name LIKE 'spp_%') AND (event.sid = 5 AND event.cid > 0) Obviously we thought this might be an issue so we rolled back without success. A search of the web didn't reveal much. Something has been introduced by that or another element that we are unabled to diagnose and cure manually. So then I thought that maybe the alert cache was confused, so I attempted to rebuild it. It failed to rebuild it. So I turned on debug mode in ACID and in PHP (log sql) and had a look at how the caching gets built. <logs omitted for brevity - if you can help I'll send them along> So then I started digging with psql into the database to see how the dupes could be invading the database. I'm a UNIX security/firewall/IDS guy, not a database guy, so bear with me. I checked out the incoming alert database and although there are duplicate cid numbers in the alert table, they are each from different Sensors (sensor 1 and sensor 5). I confirmed in the schema that the SID and CID are together uniquifiers for the event and acid_event table. So I am trying to figure out what might have happened. Our process here is that we try to keep our snortnew database cleared; we triage and archive them to our archive database once they are looked at. If they are an incident we email them out to ourselves and archive-move them. This is within the norm of what others do I think, and I haven't been able to turn up anything that would cause this to be problematic. I can see it being how we added the second sensor. It may have been a bit haphazard; you can see that because our sensors are numbered 1 and 5 :) However, the SQL indicates the problem is not with sensor 5 (most recent) but sensor 1! hcsnortnew=# select * from event where cid=2391; sid | cid | signature | timestamp -----+------+-----------+------------------------ 5 | 2391 | 31 | 2002-03-14 10:34:23-05 So then a bit later I did a hcsnortnew=# SELECT MAX(cid) FROM event WHERE sid=1; max -------- 733646 (1 row) hcsnortnew=# SELECT MAX(cid) FROM event WHERE sid=5; max ------ 2396 (1 row) Hmmm... That's pretty close to 2391! And my incoming database is being updated, so I tried to rebuild the cache again. sensor #5: event.cid = 2396, acid_event.cid = 0 Database ERROR:Database ERROR:ERROR: Cannot insert a duplicate key into unique index acid_event_pkey INSERT INTO acid_event (sid,cid,signature,timestamp, ip_src,ip_dst,ip_proto, sig_name, sig_priority, sig_class_id) SELECT event.sid as sid, event.cid as cid, signature, timestamp, ip_src, ip_dst, ip_proto, sig_name, sig_priority, sig_class_id FROM event INNER JOIN signature ON (signature = signature.sig_id) LEFT JOIN iphdr ON (event.sid=iphdr.sid AND event.cid=iphdr.cid) WHERE (sig_name LIKE 'spp_%') AND (event.sid = 5 AND event.cid > 0) INSERT INTO acid_event (sid,cid,signature,timestamp, ip_src,ip_dst,ip_proto, sig_name, sig_priority, sig_class_id) SELECT event.sid as sid, event.cid as cid, signature, timestamp, ip_src, ip_dst, ip_proto, sig_name, sig_priority, sig_class_id FROM event INNER JOIN signature ON (signature = signature.sig_id) LEFT JOIN iphdr ON (event.sid=iphdr.sid AND event.cid=iphdr.cid) WHERE (sig_name LIKE 'spp_%') AND (event.sid = 5 AND event.cid > 0) How about that... The error is on the same CID as the current highest CID number for sensor #5. I looked at the code, which indicated that BOTH SID and CID are uniquifiers for the event database but I couldn't find the same information about the acid_event database. So for now I am stuck on this. Sorry to bother you all about it but it seems like there is some sort of exceptional condition that's not handled by ACID when it creates its cache from alerts that are constructed from multiple sensors. However, knowing that many people run lots of sensors pointed at a single database, I hardly think it's the code - it's gotta be something I did. I need to get unstuck to determine what I did to create this exceptional condition. Any help is great. Thanks, Wynn _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users
Current thread:
- CID duplication issues with ACID, snort and multiple sensors W Fenwick (Mar 22)