Last CID and Duplicate Records

["Ron Shuck" <rshuck () Buchanan com>]

Hi All,

I have been experiencing a problem MySQL problem lately. When archiving
records in ACID, I would receive duplicate record errors. The
snort-users archive provided an answer to why. To summarize, when a
sensor starts it uses the max(cid) in the snort.event database table,
but due to archiving this can often be 0 or less that the max(cid) in
the snort_archive.event DB table. This causes duplicates on subsequent
archives if the sensor is restarted. 

The snort.sensor table stores the last_cid and this is updated on proper
start up and shut down. But here is the problem (IMHO). In case of a
crash, if the max(cid) is greater than the last_cid, the field is
updated. This is great and necessary. However, regardless of the value
of last_cid, the max(cid) is always used to determine the next cid to
use by the sensor. So, the last_cid field is just info, it is never
really used.

So, to correct this for my installation, I modified the database code to
use the greater of the max(cid) or last_cid. The code that updates the
last_cid if max(cid) is greater is still valid. What this does is almost
eliminate duplicate records caused by archiving. Of course, if there is
a crash duplicates could still be possible. The exception is that if
there is a crash, and you manually update the last_cid. It also means
that you can set the next cid to be used by stopping the sensor,
updating the sensor.last_cid field with the desired value. As long as
the last_cid is greater than the max(cid) all is well.
 
Modified 'output-plugin/spo_database.c' (476)
OLD: data->shared->cid = event_cid
NEW: data->shared->cid = event_cid > sensor_cid ? event_cid : sensor_cid

This was valid for 2.0.0, the line numbers may be off a bit for later
versions.

Best Regards and happy Snorting,


Ron Shuck, CISSP, GCIA, CCSE - Managing Consultant 
Buchanan Associates - A Technology Company in the People Business 


-------------------------------------------------------
This SF.net email is sponsored by: Perforce Software.
Perforce is the Fast Software Configuration Management System offering
advanced branching capabilities and atomic changes on 50+ platforms.
Free Eval! http://www.perforce.com/perforce/loadprog.html
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users