Re: "general badness" AS-based reputation system

[Tom Vest <tvest () eyeconomics com>]

On Sep 26, 2011, at 1:11 AM, Manish Karir wrote:

> On Sep 25, 2011, at 11:31 PM, Tom Vest wrote:
>
> > On Sep 25, 2011, at 9:23 PM, Manish Karir wrote:
> >
> > > On Sep 25, 2011, at 6:31 PM, nanog-request () nanog org wrote:
> > >
> > > > Message: 9
> > > > Date: Sun, 25 Sep 2011 18:37:17 +0300
> > > > From: Gadi Evron <ge () linuxbox org>
> > > > To: nanog () nanog org
> > > > Subject: "general badness" AS-based reputation system
> > > > Message-ID: <4E7F4AAD.8020400 () linuxbox org>
> > > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> > > >
> > > > Having run one of these in the past, when take-downs of C&Cs was still 
> > > > semi-useful, my ethos on this is problematic, however, I am as of yet 
> > > > undecided as to this one. An AS-based reputation system for all sorts of 
> > > > badness:
> > > >
> > > > http://bgpranking.circl.lu/
> > > >
> > > > In my opinion, third-party security based AS-reputation systems will 
> > > > eventually become de-facto border filtering systems for ISPs, but that 
> > > > day is still not here, as that is still socially unacceptable in our 
> > > > circles, and will remain so until it becomes _necessary_.
> > > >
> > > > Regardless of my musings of Operators World cultural future, this 
> > > > systems seems rather interesting, and no doubt you'd want to take a look 
> > > > at your listing.
> > > >
> > > > Gadi.
> > >
> > > We tried to outline some of the challenges of building such a system in our NANOG52 presentation:
> > >
> > > http://www.merit.edu/networkresearch/papers/pdf/2011/NANOG52_reputation-nanog.pdf
> > >
> > > In particular see slide 4. where we tried to lay down what we think the requirements are for a socially acceptable
> > > reputation system.  
> > >
> > > With a bit of luck we might be able to announce the release of our system before the next NANOG mtg, but in 
> > > my opinion collating host reputation reports is just a small and the easiest part of the effort.  The key is in 
> > > solving the challenges of allowing (and incentivizing) participation and being robust to false information
> > > injection.
> > >
> > > Comments are welcome.
> > >
> > > Thanks.
> > > -manish
> >
> > Hi Manish, 
> >
> > Looks like very interesting work.
> >
> > Does the system that you'll be announcing provide some means of coming to terms with challenges like the following?
> >
> > 1. Many large operators administer multiple ASNs, but some of the resulting AS sibling relationships may not be 
> > identifiable as such based on public-facing whois records, or interconnection relationships, or any other public 
> > data sources. Does your system incorporate some means of attributing reputation-related information at the 
> > (multi-AS) institutional level -- even when the contours of such institutions are not self-evident?
> >
> > 2. Some members of the ARIN community have recently floated a policy proposal which if approved would make ASNs 
> > transferable, and some supporters of that proposal have argued that RIR involvement in such transfers should be 
> > strictly limited to the passive recording of whatever information is voluntarily disclosed by the transacting 
> > parties, if and when a disclosure is made. Does your system ascribe reputation "strictly" to specific ASNs, such 
> > that declared changes in an ASN's "ownership"/control would not affect that ASNs accumulated reputation record 
> > to-date? Alternately, if declared changes to an ASN's ownership would affect (e.g., restart) an ASN's reputation 
> > history, will your system incorporate some mechanism for assessing the material/operational "substance" of ASN 
> > re-registration events (e.g., to filter for possible "re-registrations of convenience")? 
> >
> > I like to ask these sort of questions (for any/all proposed systems like this) because it seems to me that any 
> > system that associates increasing value with a cumulative record of consistent "approved" behavior over time must 
> > take extra care not to provide *asymmetrical* opportunities (i.e., to some but not all participants) to isolate and 
> > sanitize their own "disapproved" behavior, thereby leaving their longstanding (favorable) reputations intact. 
> >
> > Note that this problem is *not* reducible to the idea that a reputation system must be absolutely infallible. 
> > Obviously it is not reasonable to demand something that is impossible to deliver. However, it is altogether 
> > reasonable to demand that any system that is intentionally designed to produce a new, endogenously-driven 
> > reputation-based hierarchy of operational entities does something more than just recreate and reinforce pre-existing 
> > hierarchies that are completely orthogonal to "reputation."
> >
> > Look forward to hearing more!
> >
> > Regards, 
> >
> > TV

> Hi Tom,
>
> At what granularity reputation is useful is an excellent question.  Obviously we already have lots of single data 
> source based host reputation sources.
> Other possible aggregations are prefixes, ASNs, and as you suggest organizations (which might be multi-ASN).  Another 
> extreme possible aggregation is country.
>
> In my opinion BGP prefix is the right level of aggregation up to be actually useful rather than narrow host 
> reputation lists.  You might expect hosts in a 
> prefix to be under the same security policy regime and hence have similar
> likelihood of future malicious behavior so this approach would be more useful than host reputation which is entirely 
> reactive and ASN reputation which 
> does'nt allow for different parts of an organization which might run their networks with different policies.  
>
> In our system an ASN's reputation at any given time is based on aggregating up the reputation of all the prefixes 
> originated by that ASN.  Therefore it does
> not really have a reputation that exists independently of its component prefixes.  If an ASN were to be transferred 
> we would simply recompute the 
> current reputation to be based on the new set of prefixes it is originating.
>
> For prefix reputation you would want to track it on a historical basis with the assumption that it is quite unlikely 
> that a prefixes reputation will undergo a sudden change and therefore tracking historical data would be useful.  We 
> do not do this right now, but this is not a solved problem yet ;)
>
> -manish

Thanks Manish. 

If I understand your response correctly, it sounds like the proposed system treats prefix-level reputation "strictly," 
ala my (#2) questions above. Given the fact that several RIRs already permit the transfer of prefixes between different 
ASN-operating entities, it might be worth considering the same questions again with the subject "ASN" replaced by 
"prefix." If some quantity of IPv4 continues to be necessary in order for an operator to be meaningfully autonomous, 
and this condition persists for a decade or longer after the unallocated IPv4 pool is depleted (assumptions that were 
frequently cited by IPv4 transfer policy supporters), it seems reasonable to assume that the already large number of 
single (IPv4) prefix-originating ASNs will only continue to grow, as small quantities of IPv4 are transferred from 
incumbent operators with "large" IPv4 reserves to new entrants who are likely to be highly dependent on the transferred 
resources (which might carry their own lingering reputation "baggage") for interdomain connectivity. To me at least, 
that suggests that the capacity to track reputation-relevant information over time and across "ownership" changes is 
likely to become an increasingly important requirement/demand driver for reputation systems, while the applicability of 
origin-AS information averaging methods is likely to continue declining. 

Food for thought, 

TV