Re: capturing full URL information via DNS request logs

[Justin Azoff <JAzoff () ALBANY EDU>]

On Wed, Oct 09, 2013 at 08:03:02PM +0000, Youngquist, Jason R. wrote:
> Hi All,
>
> Currently we have a network monitoring device using netflow.  One problem we are having with this device is it 
> doesn't give us URL information.  There are a few other methods that were recommended to us in order to get this 
> information.  Instead of getting an IP address that points to Akamai (ie. this is want is captured via netflow), one 
> person suggested that it was relatively easy to capture the original content that the user was downloading.  Ie. in 
> the original DNS request the URL information would be included in the packet info.  Are people using DNS logs to 
> capture this type of URL traffic?  If so, does it provide the full URL, or just the DNS host name?  DNS host name 
> would be useful, but full URL would be even better.

Bro is your best bet for this.  for http you will get the full url, for
https you will get the hostname.

If you just want something like netflow records but that include
hostnames I wrote a script for bro that adds the http or https hostname
to the connection log:

https://github.com/JustinAzoff/bro_scripts/blob/master/conn-hostnames.bro

-- 
-- Justin Azoff
-- Network Security & Performance Analyst