Barnyard2 Kafka

[Jaime Nebrera <jnebrera () eneotecnologia com>]

Dear all,

The redBorder team is pleased to announce the availability of the Beta 
release of Barnyard2-Kafka plugin in our Github repository 
https://github.com/redBorder/ under GPL license.

This is an extension of Barnyard2 2-1.13 official release to add the 
following capabilities:

* Ability to send Snort events using an Apache Kafka messaging system 
(http://kafka.apache.org/)
* Preprocessing of certain Unified2 fields in order to provide enhanced 
meta data information
- Geolocation of IPs based on Maxmind libraries
- IP translation based on /etc/hosts & /etc/barnyard_networks information

In future releases we hope to extend the meta data fields provided (fe 
services information extracted from /etc/services) but for now we 
believe this is ok. This patch is usable, but beta quality, use at your 
own risk. Of course, we would really appreciate any help to extend the 
number of Unified2 fields supported as well as testing in real 
scenarios. We have based our contribution in CSV and SQL Barnyard2 plugins.

Apache Kafka is a new messaging system several orders of magnitude 
faster than AMPQ or similar. By using this framework, we will be able to 
more easily plug Snort events into a BigData environment. Just two ideas 
in this regard, it would enable to save Snort events in a Hadoop 
(http://hadoop.apache.org/) cluster as well as preprocess them using 
Twitter's Storm (http://storm-project.net/)

As for redBorder project, we are working on the real time management of 
the events for the GUI as well as a scale out capable correlation 
engine, that will not only process events generated by Snort but also 
from other elements in our framework. More information here 
(http://redborder.net/redborder-roadmap/)

Of course, we would like to thank our sponsors and clients for 
supporting us into making this public. Also, the Barnyard2 and Snort 
developers for their great software. We just hope this patch helps the 
community.

Regards

PS.- I work for the company developing redBorder

***********************************************************************************

1 Using alert_json barnyard2 plugin
*****************************
If you want to use alert_json barnyard2 plugin, you have to put it in 
barnyard2.conf file.
The format of the argument passed to the plugin is:

output alert_json: kafka://<host>:<port>@<topic>

Where host, port and topic are the kafka host, port and topic (not 
zookeepers one).

2 Host and network in readable format:
*******************************
Alert_json can can print a human readable string plus the default host 
string. For example, if you
have the hostname “foo PC” associated with the “192.168.100.3” ip in 
/etc/hosts file, alert_json will
print “foo PC” plus “192.168.100.3” and the number representation of the 
ip.

In the same way, alert_json can print the destination or source network 
of the packet. You have to make
an entry in “/etc/barnyard_networks” indicating this. For example, the 
entry “192.168.100.0/24 foo
network” will make alert_json print the network name plus the network id.

In case alert_json does not locate the network in the file, it will 
print “0.0.0.0/0” instead.

3 GeoIP
*******
Alert_json can locate the region of the IP too. You just have to have 
libGeoIP installed and compile
the sources with GEO_IP macro defined (it's defined by default). Also, 
you have to put the database
in “/usr/local/share/GeoIP/GeoIP.dat”

If you want “alert_json” print geo-localization information too, you 
have to compile barnyard with
geo-ip support:

./configure --enable-geo-ip

-- 
Jaime Nebrera -jnebrera () eneotecnologia com
Consultor TI - ENEO Tecnologia SL
C/ Manufactura 2, Edificio Euro, Oficina 3N
Mairena del Aljarafe - 41927 - Sevilla
Telf.- 955 60 11 60 / 619 04 55 18


------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!