Re: Snort Anomaly

[Kevin Ross <kevross33 () googlemail com>]

It depends what you mean by anomaly. These days "anomaly" to me means odd
HTTP communications, useragents, geolocation patterns, traffic anomalies
like bad fields for DNS or hosts talking on protocols they shouldn't be
like non-DNS servers trying to contact external DNS etc. To be more capable
of detecting these things and other anomalies I suggest taking a network
security monitoring approach with multiple levels of tools. This means
collecting various data from IDS, network etc and applying detection to it.
An excellent recently released book on this is this which while I am not
too far into it the book is truly excellent; especially as it covers snort,
anomaly detection, BRO (which very nicely complements things like Snort).

http://www.amazon.co.uk/Applied-Network-Security-Monitoring-Collection/dp/0124172083/ref=sr_1_1?ie=UTF8&qid=1389194990&sr=8-1&keywords=applied+network+security+monitoring

Obviously though you don't need a book to learn this as you can read
documentation on each of these bits. To get to a good detection level I
would suggest looking into the following things:
- Make sure you have Snort tuned so you aren't overwhelmed and the rules
and preprocessors are setup as you want them. Read the Snort documentation
on this, a lot of rules and preprocessor settings will highlight traffic
anomalies anyway.

- Install BRO http://www.bro.org/. It can detect other anomalies and also
generates very detailed logs on HTTP traffic, file hashes, tunnels, DNS,
other protocols that will complement any alerts you get from Snort etc. I
then feed those logs and IDS logs and things into ELSA
http://code.google.com/p/enterprise-log-search-and-archive/ which allows me
to do querying on all events surrounding a snort alert and also a lot of
hunting (i.e show me all unique useragents in my traffic and it will count
them up and display that, show me all executables from certain countries
etc). With snort I also have Snorby setup and full packet capture with
openfpc so it can be queried easily from Snorby from alerts. It can also
extract files from the network (which Snort 2.9.6 can do too) but the
advantage is also hashing of all files in protocols. So executables, HTML
pages, Java files, PDFs everything is getting hashed so even if you don't
have a file you can search for the hashes on things like Virustotal.

- Setup full packet capture solution like OpenFPC, Moloch or StreamDB (I
use OpenFPC due to it being integrated into Snorby and it is less intense
than say Moloch which indexes network traffic for my sensors). This allows
you to analyse the traffic in depth depending how far you can go back (1
day min 3 days ideal but you may find it is only hours. Still some FPC for
as long as your disk space allows (and you can ignore hosts, protocols etc
with BPF filters to increase that time) is better than none.

- Other types of anomaly detection can be implemented in other things such
as if you have a SIEM with your firewall logs going into it if you create a
correlation rule for high port numbers (above 1024 but not well known high
port numbers like SIP ports etc) and then log for UDP and TCP firewall
denies for so many in a certain time like a minute period you will actually
pick out P2P protocols with no knowedge of the protocol itself. I.e Using
this logic and some negation for my enviroment I reliably have detected
(although it may not have been the only alert) BitTorrent Traffic, Zeus
trojan P2P protocol and other protocols for malware etc. This will be very
useful as P2P is used increasingly in malware families.

- Another good thing is PassiveDNS ideas which you can get going with
https://github.com/gamelinux/passivedns. Just logging in with NXDOMAINs
into a database with the web interface is good and for instance you can
create a lookup in Snorby so that when you have an IDS alert you can
quickly lookup the IP in your PassiveDNS database for domains which can
very quickly help you determine a false positive or a true positive and
even when the incident first appeared. I.e I have had alerts for exploit
kits but through DNS for the other names resolved to the IP I have found
previously used domains and when they were seen and am then able to look
back and other logs at those times. Also using regular expressions,
blacklists and other methods in SIEM for NXDOMAINs for instance I can
detect malicious or suspect domains: i.e alerts for domain generation
algorithm domains (https://blog.damballa.com/archives/1504), bad domains,
supect domains such as each day I extract with a script all new domains
queried (and also cases where new IPs mapped to a name) that day and then
with some negation and other things. The logic being if that is the first
time ever it has appeared within your enterprise and it looks kind of
suspicious it just might be.

While no one thing here is a silver bullet the combination of all the
combined tools and methods is basically provided lots of ability to detect
intrusions, properly analyse them, hunt for the unknown, detect anomalies
etc. With this you will end up with:

- Snort alerting you to all kinds of intrusions and anomalies. For
anomalies though protocol rules and the preprocessors which you can read
about in the documentation is where you should look.
- BRO IDS providing detailed logging and if fed into something like ELSA,
SPLUNK, Logstash etc analytics. Also actual on disk BRO logs compress to
very little space automatically so essentially you have a historical record
of all flows, IRC chats, FTP traffic, HTTP records, file hashes and so on
for a long time of perhaps many months or even years.
- Full packet capture. Useful for short term but high detail analysis
- File extraction for analysis if you implement in BRO/Snort. You can then
do other analysis like running tools on them, checking the file hashes on
Virustotal frmo BRO etc
- PassiveDNS will allow you to analyse URLs and IPs for their relationships
and it will provide a long term historical analysis (i.e a partner
organsiation says they have malware which talks to badguys.com. Have you
been hit? You can go to that, type it in and if you get results you will
have a first and last time to begin hunting through other logs and BRO
would have even more detail. Also with regex you can detect all kind of
anomalies and if you look at research like  http://labs.umbrella.com/
http://www.lastline.com/papers/dns.pdf and
https://www.damballa.com/damballa-labs/publications.php you might get more
ideas on things in DNS to look for to detect malicious activity (or simply
feeding in blacklists of known bad ones).

Hope that helps,
Kevin


On 7 January 2014 18:38, Mr Smith <engineer.demo2020 () gmail com> wrote:

> Hi
> I Have a question about Snort:
> What is the best solution to improve Snort performance in terms of
> "Anomaly Detection" Capability?
> What is the best solution to add "Anomaly Detection" capability into Snort?
> 1. Using a Host-Based IDS(like what?) in conjunction with Snort(NIDS)?
> 2. Adding anomaly based plugins(like what) into Snort?
> 3....?
>
> Thanks
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest
> Snort news!
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!