Re: Open source Netflow analysis for monitoring AS-to-AS traffic

[Peter Phaal <peter.phaal () gmail com>]

Brian, you may want to see if your routers support sFlow (vendors have
added the feature over the last few years).

In particular, see if it includes support for the sFlow extended_gateway
structure:

/* Extended Gateway Data */
/* opaque = flow_data; enterprise = 0; format = 1003 */

struct extended_gateway {
   next_hop nexthop;           /* Address of the border router that should
                                  be used for the destination network */
   unsigned int as;            /* Autonomous system number of router */
   unsigned int src_as;        /* Autonomous system number of source */
   unsigned int src_peer_as;   /* Autonomous system number of source peer */
   as_path_type dst_as_path<>; /* Autonomous system path to the destination
*/
   unsigned int communities<>; /* Communities associated with this route */
   unsigned int localpref;     /* LocalPref associated with this route */
}

The dst_as_path field is particularly valuable since it allows you to see
who your customers are peering with.

While not a complete solution, you might want to take a look at sflowtool,
https://github.com/sflow/sflowtool, to decode the sFlow records and convert
them to JSON. It's not hard to write a Python script to calculate BGP
peering metrics and push the results into a time series database
(Prometheus, InfluxDB, etc) and build dashboards in Grafana. The following
article gives a few examples:

https://blog.sflow.com/2018/12/sflow-to-json.html

On Tue, Mar 26, 2024 at 5:06 PM Brian Knight via NANOG <nanog () nanog org>
wrote:

> What's presently the most commonly used open source toolset for monitoring
> AS-to-AS traffic?
>
> I want to see with which ASes I am exchanging the most traffic across my
> transits and IX links. I want to look for opportunities to peer so I can
> better sell expansion of peering to upper management.
>
> Our routers are mostly $VENDOR_C_XR so Netflow support is key.
>
> In the past, I've used AS-Stats <https://github.com/manuelkasper/AS-Stats>
> for this purpose. However, it is particularly CPU and disk IO intensive.
> Also, it has not been actively maintained since 2017.
>
> InfluxDB wants to sell me
> <https://www.influxdata.com/what-are-netflow-and-sflow/> on Telegraf +
> InfluxDB + Chronograf + Kapacitor, but I can't find any clear guide on what
> hardware I would need for that, never mind how to set up the software. It
> does appear to have an open source option, however.
>
> pmacct seems to be good at gathering Netflow, but doesn't seem to analyze
> data. I don't see any concise howto guides for setting this up for my
> purpose, however.
>
> I'm aware Kentik does this very well, but I have no budget at the moment,
> my testing window is longer than the 30 day trial, and we are not prepared
> to share our Netflow data with a third party.
>
> Elastiflow <https://www.elastiflow.com/> appears to have been open source
> <https://github.com/robcowart/elastiflow?tab=readme-ov-file> at one time
> in the past, but no longer. Since it too appears to be hosted, I have the
> same objections as I do with Kentik above.
>
> On-list and off-list replies are welcome.
>
> Thanks,
>
> -Brian