nanog mailing list archives
Re: Death of the Internet, Film at 11
From: Josh Reynolds <josh () kyneticwifi com>
Date: Mon, 24 Oct 2016 07:03:45 -0500
You CAN actually block things, within reason. The caveat is you simply have to disclose it. There is a 'reasonable network management' clause. IANAL, please consult your telecommunications legal team. On Oct 24, 2016 1:25 AM, "Richard Holbo" <holbor () sonss net> wrote:
I run/manage the networks for several smallish (in the thousands of customers) eyeball ISP's and I appreciate a nice "hey you've got a bot" or "someone is scanning" me notice to my abuse emails. They are useful in identifying crap that's going on, so for those of you who have the resources to do that... I appreciate it, we do read them at my networks and try to do something. That said... getting end users to actually fix the broken routers etc. etc. is NOT easy. Very often we'll notify customers, they will _take their stuff to the local computer repair guy_ ... or office depo.... and they will run whatever auto scan they have and say it's all fine. Customer puts it back in, it's still broke, and they call customer support and want us to pay for the trip because _their_ expert says it's fine... IMHO since the advent of Net Neutrality... I cannot simply block all of X, Y or Z at my edge and tell the customers it's for the best. I'd love to block some stuff in and outbound to customers, but then the customer just yells at us and files complaints with the PUC because _they have a right to it_.. So those of you calling for Government interference... we've already done that and it does not help. /rh On Sun, Oct 23, 2016 at 10:56 PM, John Weekes <jw () nuclearfallout net> wrote:On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote:... I've recordedabout 2.4 million IP addresses involved in the last two months (anumberthat is higher than the number of actual devices, since most seem to have dynamic IP addresses). The ISPs behind those IP addresses have received notifications via email...Just curious... How well is that working out?For the IoT botnets, most of the emails are ignored or rejected, because most go to providers who either quietly bitbucket them or flat-out reject all abuse emails. Most emails sent to mainland China, for instance, areinthat category (Hong Kong ISPs are somewhat better). For other botnets, such as those using compromised webservers running outdated phpMyAdmin installs at random hosts, harnessing spun-up services at reputable VPS providers (Amazon, Microsoft, Rackspace, etc.), or harnessing devices at large and small US and Canadian ISPs, we have had better luck. Usually, we don't hear a response back, but those emails are often forwarded to the end-user, who takes action (and may ask us forhelp,which is how we know they are being forwarded). The fixes can enough to reduce attack volumes to more manageable levels. Kudos go out to the large and small ISPs and NSPs who have started policing SSDP and other reflection traffic, which we also send out some notifications for. In some cases, it may be that our emails spurred themtonotice how much damage those attacks were doing and how much it wascostingthem to carry the attack traffic. I've tried this myself a few times in the past, when I've found thingsthat appear to be seriously compromised, and for my extensive trouble I've mostly received back utter silence and no action. I remember that after properly notifying security@ some large end-luser cable network in the SouthEast (which shall remain nameless) I got back something along the lines of "Thank you. We'll look into it." and was disgusted to find, two months later, that the boxes in question were still utterly pwned and in the exact same state they were two months prior, when I had first reported them.We do get our share of that, as well, unfortunately, along with our share of people who send angry responses calling the notifications spam (I disagree with them that sending a legitimate abuse notification to a publicly-posted, designated abuse account should be considered spam) orwhoflame us for acting like "internet police". But, we persist. Some people change their minds after receiving multiple notifications or after we explain that DoS traffic costs them money and hurts their customers, who will be experiencing degraded service and may silently switch providers over it. I guess that's just an example of what somebody else already noted here,i.e. that providers don't care to spend the time and/or effort and/or money necessary to actually -do- anything about compromised boxes, and anyway, they don't want to lose a paying customer. So, you know, let's just say for the sake of argument that right now, today, I know about a botnet consiting of a quarter million popped boxes, and that I have in-hand all of the relevant IPs, and that I have no trouble finding contact email addresses for all of the relevant ASNs. So then what?I use scripts to send out an abuse notification to some percentage of the compromised hosts -- the ones sending some significant amount of the traffic. The notification includes a description of what we saw and timestamped example attack traffic, as interpreted by tcpdump. If further traffic is seen later from the same host, another notification will be sent, after a cool-off period. The emails are plain text and we don't try to use them as advertisement. We also don't force a link to be clicked to see more details or torespondback. I don't like to receive such emails myself and have found thatthosetypes are more likely to be ignored. The question is: Why should I waste my time informing all, or even anyof these ASNs about the popped boxes on their networks when (a) I am not their customer... as many of them have been only too happy to gleefully inform me in the past... and when (b) the vast majority simply won't do anything with the information?I'm not saying that everyone should send abuse notifications like we do, since it can be a big task. But, in response to someone wondering iftheirnetwork is being used for attacks, or asking how they could help topolicetheir own network, I am saying that making sure that inbound abuse notifications are arriving at the right place and being handled appropriately is important. And while we are on the subject, I just have to bring up one of mybiggest pet peeves. Why is it that every time some public-spirited altrusitc well-meaning citizen such as myself reports any kind of a problem to any kind of a company on the Internet, the report itself gets immediately labeled and categorized as a "complaint". If I spend some of -my- valuable time to helpfully try to let somebody else know of a problem on their network, or with their web site, and if that report gets categorized as a "complaint" then what does that make me? A "complainer"?? I don't need this kind of abuse and denegration from people who I'm trying to help. Like most other people, if I am in need of some personal denegration and abuse... well... I have relatives for that.There's a spectrum of people responding to these and some percentage are just jerks, as in real life. But, I like to think that the majority of at least NA providers are represented by professionals who just don'trespondout of courtesy because they don't want to flood our inboxes with simple acknowledgements. Those of us experiencing these attacks appreciate the community support, both from people like you who also send notifications and those whohandlethe notifications on the receiving end. -John
Current thread:
- Re: Death of the Internet, Film at 11, (continued)
- Re: Death of the Internet, Film at 11 bzs (Oct 23)
- Re: Death of the Internet, Film at 11 John Weekes (Oct 22)
- Re: Death of the Internet, Film at 11 Ronald F. Guilmette (Oct 23)
- Re: Death of the Internet, Film at 11 Stephen Satchell (Oct 23)
- Re: Death of the Internet, Film at 11 David Conrad (Oct 23)
- Re: Death of the Internet, Film at 11 Stephen Satchell (Oct 23)
- Message not available
- Re: Death of the Internet, Film at 11 Larry Sheldon (Oct 23)
- Re: Death of the Internet, Film at 11 John Weekes (Oct 23)
- Re: Death of the Internet, Film at 11 Richard Holbo (Oct 23)
- Re: Death of the Internet, Film at 11 Jean-Francois Mezei (Oct 23)
- Re: Death of the Internet, Film at 11 Josh Reynolds (Oct 24)
- RE: Death of the Internet, Film at 11 Emille Blanc (Oct 24)
- Re: Death of the Internet, Film at 11 Ronald F. Guilmette (Oct 25)
- Re: Death of the Internet, Film at 11 bzs (Oct 25)
- Re: Death of the Internet, Film at 11 Aaron C. de Bruyn via NANOG (Oct 24)
- Spitballing IoT Security Ronald F. Guilmette (Oct 24)
- Re: Spitballing IoT Security Jared Mauch (Oct 24)
- Re: Spitballing IoT Security Matthias Waehlisch (Oct 24)
- Re: Spitballing IoT Security Jared Mauch (Oct 25)
- Re: Spitballing IoT Security Ronald F. Guilmette (Oct 25)
- Re: Spitballing IoT Security Jean-Francois Mezei (Oct 25)