: Why it is difficult to counter spam

[Dave Farber <dave () farber net>]

___

Dave Farber  +1 412 726 9889



...... Forwarded Message .......
From: Dave Crocker <dcrocker () brandenburg com>
To: Dave Farber <dave () farber net>
Cc: Interesting People <ip () v2 listbox com>
Date: Wed, 22 Sep 2004 09:56:47 -0700
Subj: Why it is difficult to counter spam

Dave,

As much as we all would like world peace, no more cockroaches and the end 
of 
crime, some problems are not so easy to solve. Spam is a social problem, 
rather 
than a "failure" of the technology. Spam conforms to Internet technical 
standards. Responding to the rise in spam is responding to a change in 
requirements. Worse, spam is not monolithic.  At a minimum, we need to 
distinguish between overly aggressive marketing, from otherwise-responsible 
business, versus criminal conduct by rogue operators. 

The  occurrence of Internet spam is very much like having a small town turn 
into 
a big (American) city.  We have changed from a safe, accountable 
environment, to 
one that requires more caution and more formal processes. We need to add 
locks 
to our doors.  We need to be careful when talking to strangers.  We need to 
show 
our identification when cashing a check.

> From a technical standpoint, spam looks pretty much like legitimate mail.  
Some 
unsolicited mail is essential to the conduct of human affairs. Some bulk 
mail is 
good, such as subscription-based lists like Interesting People. Some 
commercial 
mail is good, such as purchase order confirmations.  So the first difficult 
question is how to distinguish spam?

The second difficult question is how to institute spam control techniques 
without causing fundamental, long-term damage to the utility of email.  The 
current 90+% traffic load of spam is doing its own damage, of course, but 
we 
need to be careful that we do not fix one problem by causing another.  
Worse, we 
need to be careful we do not fix transient symptoms by making long-term 
alterations.

So far, the spamming community is proving to be better organized, more 
intelligent and more aggressive than the anti-spamming community.  The 
typical 
architecture of a spamming system is remarkably sophisticated, involving a 
multi-level, globally distributed hierarchy of millions of machines. 

So it is well and good to say that the current problem is major and that we 
all 
must therefore accept some changes. However would be irresponsible to make 
basic 
changes to an essential, global infrastructure service, without having a 
clear 
understanding of the impact of those changes and a clear consensus that the 
impact is acceptable.   

Such analysis and consensus has been notably lacking from public discourse 
about 
spam. Most public discussions involve emotion, politics and opportunism... 
just 
like any other public policy exchange.  

For all of that, serious work very much is being pursued.  Various national 
legal efforts around the world are happening, but laws are not instruments 
of 
rapid intervention or surgical precision.  Worse, we simply do not know for 
certain what laws will work -- The dictum that the Internet routes around 
barriers is true for spammers, too. So the current round of legal efforts 
constitute experiments. Over time, I expect things to settle on some common 
templates.  At a minimum, they will establish common terminology and useful 
constraints on responsible business.  It is less clear how much effect they 
will 
have on criminal spammers.

On the technical side, there are numerous proposals for adding different 
types 
of authentication to Internet mail. They differ both in techniques and 
focus. 
For example, some authenticate the author, some authenticate the bounce 
address 
and some authenticate the mail server operator. Some combination of them is 
likely to be necessary.  In fact IETF considerations include 3 
authentication 
specifications, from Microsoft (Caller-ID/Sender-ID), Yahoo (DomainKeys) 
and 
Cisco (Identified Internet Mail), so it is difficult to say that major 
players 
are not working on this seriously.  (For reference, I am involved with two 
other 
specifications -- CSV and BATV. There are a number of others.)

However, authentication does not prevent spam.  At a minimum, we need to 
add 
accreditation (reputation) mechanisms before we are likely to make any 
serious 
inroads.  That won't "eliminate" spam, but it is likely to help. 

Unfortunately, email accreditation involves primarily social issues and we 
have 
no Internet-scale experience with it. One can debate about the 
applicability of 
various global, financial analysis and authorization services, but my main 
point 
is that the open Internet has no experience with any of this. Hence the 
adoption 
of such a capability requires extended consideration, no matter how quickly 
we 
all might wish to move. I am hoping that any standards effort for 
accreditation 
starts with sometime modest and straightforward, to reduce the time needed 
to 
get it into the field.  The more modest a standard, the easier it is to get 
it 
adopted... as long as it does something useful.  

For example, CSV defines an almost trivial mechanism for querying an 
accreditation service and receiving a yes or no.  Trivial, but we think it 
likely to be useful. At the least, this will let receiving SMTP servers do 
real-time queries, much like obtaining an ATM card approval, using a 
standard 
interface. And it will let approval-oriented accreditation services compete 
in 
an open market.  But all of this is very new territory and the technical 
standards debates have not even begun.

I'll end by noting that the operational side of spam control requires 
on-going 
collaboration among service providers and even governments. More of this is 
happening.  For example, the Chinese government and Chinese service 
providers 
have made major inroads. As of 5 months ago, China was the dominant source 
of 
spam-sending engines and spam-serving web sites. As of this month, they 
aren't. 
This is a direct result of their initiatives.

d/
--
Dave Crocker  <mailto:dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking  <http://brandenburg.com>





-------------------------------------
You are subscribed as interesting-people () lists elistx com
To manage your subscription, go to
  http://v2.listbox.com/member/?listname=ip

Archives at: http://www.interesting-people.org/archives/interesting-people/