Re: flow analysis for juniper devices

[Paolo Lucente <pl+list () pmacct net>]

On Sun, Nov 14, 2010 at 11:32:10AM -0600, Richard A Steenbergen wrote:
> On Sun, Nov 14, 2010 at 08:59:33AM +0000, Paolo Lucente wrote:
> > On Sat, Nov 13, 2010 at 09:17:55PM -0600, Richard A Steenbergen wrote:
>
> Remember that every RIB in your network can and will have a unique best 
> path selection (thanks to the EBGP > IBGP rule if nothing else), and if 
> you have a network of any size at all you'll probably have to deal with 
> multiple exits to the same network. Even if you were only concerned with 
> analyzing external traffic, you'd still need to collect a RIB per edge 
> router using an IBGP feed. 

Agree. 

> In my network this would put you well over 10 million paths, and consume
> several gigs of ram, not to mention the load of doing the routing lookups
> themselves. 

If the collector is able to keep up with the pace of the export protocol,
it will not get killed by a couple of BGP lookups per flow/sample. About
the memory figure: reducing information overlap among the RIBs results in
storing a full BGP table in few tens of MBs. On 10M paths that translates
in less than a GB of memory.

> analysis inside your network you'd need a feed from every router, and 
> maybe even active participation in your IGP. 

This is separate thing. Relying only on telemetry information for such a
task is one of the possible approaches - perhaps the quicker, if enabled
ad-hoc for troubleshooting, but not necessarily the smarter. 

> It CAN be done, but it's not pretty, and I don't think any existing free
> software has been tested under these kinds of conditions.

Define pretty: is it pretty to move control-plane information over and
over via NetFlow or sFlow? But most importantly: why passing the concept
challenging conditions is something free software should be run far away
from?

> So when a vendor says "we support sFlow", make sure they actually 
> support the message types and fields you need. :)

Indeed, agree.

Cheers,
Paolo