Re: Minemeld

[Vincent Zhen <vincent.zhen+educause () NYU EDU>]

Hi Bryan,

I have been doing a decent amount of work here at NYU with Palo Alto's Minemeld. I've had a good experience with it 
overall. Here's a quick list of the things I'm remembering off the top of my head:

+ It was relatively easy to set up on a VM
* I would note that the more IOCs you're expecting to get, the more resources the VM will need. My notes say that you 
will need upwards of 16GB of RAM and 4 cores of CPU. I'm using ~7GB/25GB of storage space for ~220K IOCs.
- If you have custom miners, updates will likely break them and you'll have errors. I've sort of rectified this by 
using the Puppet configuration management software to make sure the files are always in the right place and contain the 
right things.
+ Miners are written mostly in Python. I'm a Python guy so it was certainly happy circumstances.
- Management and organization of nodes is tedious
* The documentation is pretty good but they're sort of hard to find
+ REST API available and with varying degrees of detail

The miners we are using are:
Built in:
alienvault.reputation           
dshield.block           
ETOpen.blockIPs         
ETOpen.compromisedIPs           
ransomwaretracker.RW_IPBL               
spamhaus.DROP           
spamhaus.EDROP          
stdlib.listIPv4Generic

Custom:
<custom CIF feed 1>
<custom CIF feed 2>
<custom miner for vendor threat intel feed>
(* I don't want to divulge the custom miners I've built just in case it's not something that's supposed to be public)

We are currently using Minemeld by having it collate threat feeds, using a cronjob + Minemeld REST API -> Snort rules 
-> up-to-date IOCs in our SIEM. 
We are getting a set of NGFWs that pair very well with Minemeld. The integration- so I'm told- would be seamless. We 
are planning on using Minemeld to collate threat feeds and sending the collated feed to the NGFWs which would create a 
block list. 

NB: The true positive rate of generating dynamic block lists this way is entirely dependent on the mined threat feeds. 
If a threat feed contains a lot of false positives, your dynamic block feed will block a lot of things that will result 
in false positives. 

Let me know if you have any other questions! Minemeld was a pet project for me but will likely be put into a production 
level role in the future (happy circumstances abound!).

Regards,
Vincent
NYU IT Office of Information Security