Re: NAT IP and Google

[Derek Andrew <Derek.Andrew () usask ca>]

As others have said, Google's abuse systems are smart enough to understand
NAT and proxies, and won't block on request volume alone.  When we
automatically apply a block, we'll generally offer a captcha to give
innocent users a workaround and limit the annoyance until the abuse stops
and the block can expire

This failed at our site. Our entire IPv4 and IPv6 addresse blocks received
captcha after captcha after captcha, forever and ever.

There was a link on the page to get more information, but all that got was
another captcha.

Normally I am 100% behind Google in everything, but sadly, this has now
fallen to 99.8%.

derek




On Wed, May 21, 2014 at 10:42 PM, Damian Menscher <damian () google com> wrote:

> On Tue, May 20, 2014 at 7:21 AM, Pui Edylie <email () edylie net> wrote:
> > May I know what is the best approach so that Google would not ban our
> > Natted IP from time to time as it suspect it as a bot.
>
> As others have said, Google's abuse systems are smart enough to understand
> NAT and proxies, and won't block on request volume alone.  When we
> automatically apply a block, we'll generally offer a captcha to give
> innocent users a workaround and limit the annoyance until the abuse stops
> and the block can expire.  While we do everything we can to limit the
> collateral damage, if your organization has an infected user spewing abuse,
> you need to take responsibility for your network.
>
> IPv6 is the best long-term solution, as this will allow Google's abuse
> systems to distinguish between your users and block only those violating
> the ToS.  Please give each user a distinct /64 (this seems obvious, but
> I've seen someone put all their users in the same /96).
>
> If you can't deploy IPv6 yet, some other suggestions:
>   - Put your users behind a proxy that adds the X-Forwarded-For header with
> the user's internal IP.  Google's abuse systems use that header to limit
> blocking when possible.
>   - Review your machines for signs of infection -- many blocks are
> triggered by botnets that are sending abuse.  Another common cause is a
> browser extension that automatically sends requests.  Finally, don't set up
> monitoring to test whether you're being blocked -- those automated
> monitoring requests are also a violation of the ToS and only increase the
> chance of being blocked.
>   - If you have a proxy, test it to ensure it's not an "open" proxy.  Open
> proxies are frequently abused, and will get blocked as a result.
>   - Partitioning users across different IPs can help contain the collateral
> damage when one user's machine goes rogue.  If you load-balance all users
> across all your IPs then it will likely just result in the entire pool
> being blocked.
>
> Is there any official channel from Google which we could work with them for
> > resolution?
>
> There's no official channel for working to resolve a blocking issue.  Years
> of experience proves the abuse systems are very accurate (and constantly
> being improved) -- false positives are extremely rare.  Despite this
> certainty, due to privacy concerns no evidence can be shared back to the
> ISP to point to the source of abuse.  Since nothing can be shared except
> for times abuse was seen (which is rarely helpful due to lack of logging by
> the ISP), the response is generally just the suggestions listed above.  The
> blocks will expire on their own once the abuse has been stopped.
>
> Damian
> --
> Damian Menscher :: Security Reliability Engineer :: Google



-- 
Copyright 2014 Derek Andrew (excluding quotations)

+1 306 966 4808
Information and Communications Technology
University of Saskatchewan
Peterson 120; 54 Innovation Boulevard
Saskatoon,Saskatchewan,Canada. S7N 2V3
Timezone GMT-6

Typed but not read.