Re: My [Phil Karn] position on Comcastidiocy

[David Farber <dave () farber net>]

________________________________________
From: Dave Crocker [dhc2 () dcrocker net]
Sent: Monday, January 21, 2008 3:15 PM
To: David Farber
Cc: ip; Phil Karn
Subject: Re: [IP] My [Phil Karn] position on Comcastidiocy

Dave,

Lots of posts on this thread that worthy of response, but Phil's seems to
nicely focus on underlying issues:


David Farber wrote:
> From: Phil Karn <karn () ka9q net> Date: January 19, 2008 8:04:51 PM EST
...
> A few comments on port 25 blocking:
>
> Everybody seems to assume that blocking direct usage of port 25 somehow
> stops spam.


Folks who work in the anti-spam arena are careful *not* make this claim.

The typical mantra amongst that community is that spam and abuse are here to
stay, just like the considerable array of other social misbehaviors (eg, crime.)

The goal, then, is to bring it down to a tolerable level. Any honest effort is
acknowledged to be incremental and imperfect. As with any other effort to
control misbehaviors, different techniques are needed, with different degrees
of complexity, cost, impact and persistence (of benefit).

One of the core challenges with controlling abuse is establishing
accountability.  If you cannot determine who actually did something, there
ain't much recourse.  Port 25 has a history of being used both for
inter-organization mail relaying (which is what SMTP is classically known for)
but also for first-hop transfers (that is, initial posting from the user to
the relay service.) Given the history of the Internet, this first hop
initially did not require user authentication.

The Submission port (587) is for a service that runs what is essentially SMTP,
but it requires authentication.  Although a formal standard for some years,
Submission had poor adoption.  We finally developed a formal recommendation
(Best Current Practice - BCP) for its use that was adopted by the IETF last
year.[1]  This past week I heard about a mail user agent that (finally)
defaults to 587 rather than 25 for submission.

The BCP acknowledges that outbound port 25 blocking is done, but does not take
a position on it.  We chose to focus strictly on promoting the use of the
authenticated port 587, prohibiting its being blocked by outbound ISPs while
reinforcing that authentication for the use of the port is required at the
target server.

Frankly, we did not see the need to engage in the controversy about port 25
blocking.

My personal view is that blocking outbound port 25 is offensive but necessary.
It is offensive because it is a broad-spectrum act of control by an
intermediary.  Once an intermediary does blocking of any sort, for one
reason, the odds of their doing it for another seem to go up.  However we are
not likely to have any success at retrofitting sufficient accountability on
port 25 when it is used for initial posting.  The only option that leaves us
is with the Draconian act of blocking.

ISPs vary in their policies about the imposition of blocking, but since it is
the ISP that catches a raft of grief when it is seen as the source of abusive
email, it is the ISP that is pragmatically forced to take action.  Some ISPs
attempt to be passive and precise, by imposing a proxy server on port 25.
These are not explicitly visible to senders and do per-message inspections; on
the whole, proxies are problematic because it is difficult to do a perfect job
of emulating the port 25 channel, nevermind also trying to detect problematic
messages.

Some ISPs are more narrow-band.  They automatically restrict use of port 25
but will turn it back on for particular customers.  This is actually an
attempt to be friendly to customers, but it invites support problems.

Other ISPs have a simple, universal rule, which is to prohibit all outbound
port 25.  Period. Given the available of the Submission port (587), the
argument is that blocking port 25 is inconvenient -- since mail user agents
typically default to using port 25 -- but not fatal.

The other 'narrow-band' approach is to have port 25 automatically be *open*
for customers, unless abuse is reported on their use of it. At first blush,
this is by far the most friendly approach, but this thread on IP shows just
how problematic it can be.  Detecting abuse is heuristic and therefore
sometimes wrong.

I've always been taught the key to good customer service is predictability.
(Being predictable and helpful is better than being predictable an unhelpful,
of course, but there is actual psych research showing that predictability is
more important.)  On reflecting about the current thread, it looks as if at
attempt to have the smallest impact on user freedom actually has backfired
into a support nightmare.

Go figure.


> Exactly HOW does forcing outbound mail to take an unnecessary hop through
> the ISP's outbound MTA stop spam? Does the MTA have some sort of magic spam
>  recognizer? If so, why can't it be used by every inbound MTA?

Operations is about operators.  The ISP much more likely to have competent
operators than is a stray mass-market customer.  They are more likely to have
7x24 monitoring and more likely to have invested in more powerful anti-abuse
mechanisms.

Anyone doubting the reality of this view should consider the percentage of
mass-market user machines that are currently believed to be compromised,
versus the likely percentage of ISP mail servers...

The average user does not know anything at all about running anti-abuse
software.  Some do, but the vast majority do not.  It is therefore the job of
the ISP to provide protection services.

Should a provider also allow for expert customers?  Sure.  But that reduces to
a question of support costs, since we are now talking about additional
overhead for the ISP.


> The closest thing we have to a magic spam recognizer is Spam Assassin. It
> (or an equivalent package) is ALREADY in use by nearly every inbound MTA.
> How does duplicating this function in an outbound MTA -- or even *having*
> mandatory outbound MTAs -- help the spam problem?

The holy grail among the anti-abuse community is eliminating abuse at its
source.  The cost of catching it at the destination is enormous, so why not
try to move it to its source.

And by the way, with abuse mail consuming about 90% of the email traffic,
that's a heck of a lot of wasted network bandwidth, nevermind wasted
processing overhead at the receiver.


> Forcing outbound mail through a common MTA actually makes it MORE difficult
>  for the receiving MTAs to handle spam. Were the mail to come directly from
>  the sender's IP addresses, blacklists of offending addresses can be
> maintained.

An appealing view, but not one supported by experience.  Receive-side blocking
based on IP Address blacklists is the best we have had available for years,
but it is highly problematic.  All sorts of reasons.  Some technical, some
social.  Note, for example, that most abuse mail comes from machines owned by
legitimate users, whose machine has been compromised. So even the suspect
machines show split-personality sending behaviors.


> Let's assume a magic spam recognizer exists. Assume further that we
> consider it mandatory at the originating ISP. But its implementation in a
> mandatory outbound MTA STILL does not follow!! The magic spam recognizer
> could passively monitor end-to-end connections to port 25. They would
> proceed unhindered unless spamming is detected.

That appears to be exactly the mode that triggered this thread. So maybe it is
not a perfect choice?


> MTAs are infamous resource hogs. Users often complain of long delays in
> their outbound mail, as can be expected when everyone is forced to use them
>  for no good reason. Indeed, users should be active ENCOURAGED to minimize
>  their use of ISP resources by delivering their mail directly to its
> destinations when possible.

Survivalist models of Internet self-reliance are appealing but there is no
indication that they are practical for the mass-market of users.

d/

ps. I haven't mentioned Comcast in this note, since I think the issues are
more general.  But they are the provider that triggered this thread, and just
in case anyone worries about conspiracy theories, I'll mention that I do a
tiny amount of consulting for the Comcast email folks. However I am sending
this on my own initiative, am expressing my own views, have not contacted them
about the note and have not based it on any particulars that I might know
about their operation.


[1] C. Hutzler, D. Crocker, P. Resnick, E. Allman, T. Finch, Email Submission
Operations: Access and Accountability Requirements, RFC 5068, November 2007.

--

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

-------------------------------------------
Archives: http://v2.listbox.com/member/archive/247/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/247/
Powered by Listbox: http://www.listbox.com