more on 4 more on we are all blacklisted

[Dave Farber <dave () farber net>]

> Date: Fri, 05 Sep 2003 13:52:35 -0400
> From: Bob Frankston <Bob2-19 () bobf frankston com>
> Subject: RE: [IP] 4 more on we are all blacklisted
> To: dave () farber net, ip () v2 listbox com
> Cc: David Reed <dpreed () reed com>
>
> I don't want to repeat myself but I feel I must try to respond.
>
> In http://www.frankston.com/?name=AOLHasLeft I address some key
> misunderstandings. (I've left ip () v2 listbox com on the address list since I
> still get bounces from dave () farber net even though it may be going through
> but I don't want to repeat the "receipt" experiment again)
>
> 0. (Inserting this after writing the rest) We should assume that all letters
> are encrypted. The idea that the post office can and should look inside all
> of my mail to see if it is approved is a very very bad idea.
>
> 1. There's a big difference between an individual having a bad policy and an
> ISP forcing a policy on its users, especially a megaISP like AOL. As I point
> out in the essay, it's like a state (in the US) using its influence to
> censor textbooks. Blacklisters and other meddlers just make email
> increasingly perverse by making it dependent upon accidental properties like
> my route. How the hell am I supposed to configure my PDA -- what mail
> address can I use that simply works?
>
> 2. The idea of ISPs imposing rules on the bits you can send strikes at the
> fundamentals of the Internet - instead of getting connectivity, we simply
> have the maze of twisty passages that made the "smart networks" so
> problematic. It was only by removing these impediments that we could
> innovate and create email and the web. SMTP is a point to point protocol
> with relaying as an option. ISPs that prevent that are simply not giving you
> Internet access -- they are simply providing mindless entertainment and
> shopping. Given that you have no effective choice because these providers
> are often taking advantage of special privileges I feel you have an
> obligation to defy the restrictions. They fall into the class of CYA rules
> for the ISPs and only enforce it if it suits their purpose and often
> advertise uses that go against these restrictions such as banning webcams
> but featuring them in ads. I saw a Comcast ad telling you that you want
> their broadband connection because it allows you to listen to your music
> anywhere you are.
>
> 3. Stopping spam at the source sounds nice in theory but what is spam
> anyway? Just because we have "obvious" spam doesn't mean that all offers for
> refinancing your house are spam. Or letters from an ex might or might not be
> spam depending on my mood. Spam is defined at the edge though I can advise
> and ask others to act on my behalf.
>
> 4. A big problem, as noted, is that people have a bad conceptual model of
> what email is. I try to explain it in http://www.frankston.com/?Name=smtp
> but presume those who need to know won't read it. The header/envelope
> distinction is confusing one -- even in the world of paper mail. MCI's mail
> system got that wrong in its protocol which became obvious when I did Lotus
> Express. Unfortunately this misunderstanding goes far deeper and perhaps
> this should be point #1. People assume that spam is an intrinsic property of
> a message or the messenger and thus believe that we can have policies that
> just outlaw that stuff. The idea of intrinsic meaning in bits and atoms is
> central to so much but we tend to gloss over it. The Internet and spam force
> the issue. It's not new -- nasties that evolve immunity to a vaccine are
> another example but our language makes it sound as if a creature changes
> rather than making it clear that those that survive just take over. Same for
> spam -- we'll just call it advertising or advice. This is the problem with
> blacklisting -- and it's exacerbated by the fact that the IP address is
> volatile and, unfortunately as I keep ranting about, so is the DNS entry.
>
> 5. How bad is the spam problem really? OK, skimming through a thousand
> messages is a pain but I expect that the tools that help me will indeed
> improve and the good ones will do a lot more in helping me manage the
> thousands of unread nonspam letters I have lying around. That's creating
> value and not just fighting a losing battle. People whine about the traffic
> load when it's the attention load.
>
> 6. It could be worse, imaging having your 800 # published on a popular site.
> The namespace is dense so it happens by accident all the time. It's happened
> to be by accident -- how much worse if it were malicious. The price of the
> call is totally artificial and thus the cost of creating a call that is
> charged to the called party is basically zero but there is no recourse!
>
> Spamfixation is a serious problem and that's the real issue created by spam.
>
>
> -----Original Message-----
> From: owner-ip () v2 listbox com [mailto:owner-ip () v2 listbox com] On Behalf Of
> Dave Farber
> Sent: Friday, September 05, 2003 10:53
> To: ip () v2 listbox com
> Subject: [IP] 4 more on we are all blacklisted
>
>
> >Date: Fri, 05 Sep 2003 10:07:52 -0400
> >From: "David P. Reed" <dpreed () reed com>
> >
> >
> >Ed gerck makes a great point.  They use us against ourselves.   Here's a
> >bigger view that he inspired me to jot down.
> >
> >Spam and virus/worm attacks are evolutionary systems.   We are the
> >evolutionary landscape, and we co-evolve with them.
> >
> > From a systems point-of-view, their goal transcends ours.   We focus on
> > "securing email" against certain kinds of attacks.  But our focus on
> > specific means is their advantage.  The spammer's goal is to appear in
> > front of eyeballs on a mass scale at low cost - not specifically to
> > exploit exploitable holes like relays or anonymous sourcing.   So we end
> > up patching all the holes while the boat is sinking from new kinds of
> > holes.  Better to install some flotation foam, and quick, to stymie their
> > real goal of permeating our ship with water.  The virus/worm guy is
> > focused on "gentile terrorism" - all they want to do is to annoy/destroy
> > at minimal risk to themselves.  Again they enlist our help in destroying
> > ourselves by tricking us into blocking legitimate communications.
> >
> >We help both sets in their evolutionary race by fighting them, especially
> >by fighting them inefficiently.   Just like weak vaccines and weak
> >antibiotics enhance the evolutionary machine for "germs", weak responses
> >just train a new generation.
> >
> >It's really worth thinking about whether we are transforming low-grade
> >irritants into highly virulent and damaging strains by our ineffective
> >responses, which just serve as training sets for their evolutionary
> algorithms.
> >
> >Our strategy needs to be much more focused on the evolutionary system, not
> >on low-level patches after-the-fact.
> From: Rich Kulawiec <rsk () gsp org>
>
> On Fri, Sep 05, 2003 at 01:45:11AM -0700, Ed Gerck wrote:
>  > Thus, adding more
>  > and more ad hoc blacklists (IP and DNS based) is simply making more
>  > and more likely that an important message will NOT be received --
>  > irrespective of your efforts.
>
> 1. DNSBLs are not "ad hoc".  Most of them are quite thorough.  (Those which
> are not aren't widely used, for precisely that reason.)
>
> 2. Regardless, if you feel this way about them, then YOU should not
> be using them.  However, those of us who understand them and find
> them useful and accurate will no doubt continue to do so.
>
>  > 1. "Content-sensitive filters ignore the putative sender (since both
> spammers and
>  > virus/worms frequently forge it) and focus on the content." This is just
> half the
>  > story. The fact is that content-sensitive filters such as mailwasher do
> use the
>  > content to blacklist a sender's address.
>
> Mailwasher is an amateurish hack which includes an auto-abuse feature of its
> own -- e.g. the bounce-to-sender feature, which enables anyone naive enough
> to think that senders aren't forged to abuse another (almost certainly
> innocent) person.
>
> This is why the consensus among knowledgeable anti-spammers is that nobody
> should be using Mailwasher.  (A few minute's research with Google will
> easily bear this out.)  If you want to see a professional-grade
> content-sensitive filter, then go look at SpamAssassin.
>
>  > And just talk to most users -- who simply
>  > set their mail client filters to blacklist entire domains (*@aol.com)
> and even
>  > entire TLDs (*.cn, *.ru, etc.), irrespective of content.
>
> 1. I'm sorry that some users have decided to misconfigure their mail clients
> in broken ways because they don't understand the spam problem and proper
> strategies/tactics for dealing with it.   But this is not my problem.
>
> 2. It is a long-established principle of spam fighting that the best place
> to block spam is that place which is nearest the source.  Ideally, this
> means convincing the spammer-hosting ISP to immediately and permanently
> remove their spamming parasites without prior notice.  The next-best place
> is that the receiving MTA.  Blocking in the end-user's client is largely
> ineffective and is very much a close-the-barn-door-after-the-horse solution.
>
>  > 2. "DNSBLs [RHSBLs] ... completely IGNORE the putative sender and focus
>  > solely on the originating IP address." The simple fact that we do
> receive many
>  > 100's of spam/viruses/worms daily, and #3 below, should speak for the
>  > DNSBLs' lack of usefulness. It is not working.
>
> This is an amazing leap of total illogic.  You have failed to mention what
> DNSBLs *your* mail service provider is using, and in precisely what way
> they are used.  You have also failed to recognize that spams/viruses/worms
> may be coming FROM YOUR OWN ISP, in which case they can hardly be expected
> to block mail traffic from themselves.  And you have blithely generalized
> from YOUR experience to the experience of others -- I guarantee you that
> they are not the same.
>
> I suggest that you need a basic grounding in the theory and practice of
> DNSBLs before proceeding further.  It's obvious to me that you don't
> understand how they work, and are therefore completely unprepared to
> evaluate their advantages/disadvantages.
>
>  > 3. "...block SMTP connections from dynamic dial-up IP ranges and from
>  > residential Cable/DSL connections".  In other words, let's break the
> peer-to-
>  > peer Internet, let's make some IPs more routable than others, let's
> create
>  > "second-class" Internet users and make them 100% dependent on ISPs
>  > for SMTP.
>
> 1. You are apparently unaware that the contractual agreement between nearly
> all
> ISPs who provide dial-up/cable/DSL and their customers REQUIRES that those
> customers route their outbound SMTP traffic through the ISP's own mail
> servers,
> and thus there should never be any outbound port 25 from those network
> blocks.
>
> 2. You are also apparently unaware that some of those ISPs block outbound
> port 25 at the router level, in order to enforce #1.  This has been a common
> practices for years.
>
> 3. You are also apparently unaware that at the moment, the dial-up/cable/DSL
> networks of a large number of major providers (e.g. Comcast, Verizon,
> Roadrunner,
> Charter, etc.) are heavily infested with end-user systems that have been
> hijacked
> via SoBig variants and are being used as a massive distributed
> spamplifier.  The
> failure of these users to adequately secure their own systems, combined with
> the failure of their irresponsible ISPs to take action when duly notified of
> same, leaves the rest of the Internet with two choices:
>
>          a) Do nothing about it, and be subjected to a torrent of spam.
>          b) Block them as they're detected or pre-emptively en masse.
>
> Most sensible people have done (b).  People who chose (a) are no doubt
> drowning in spam, but that's certainly their choice.
>
>  > "Black hole" lists look a lot like a "vigilante" solution [....]
>
> 1. I suggest that you are in no position to analyze them, or their impact,
> because you obviously do not understand how they work.  You need to read
> -- for a minimal introduction -- the FAQs I referenced in my previous
> message.   You should also read the FAQs those reference, and so on,
> until you have at least a rudimentary understanding of the current state
> of spam/anti-spam strategies.
>
> 2. There is nothing in the least bit" "vigilante" about deciding how
> one's own systems and networks are to be used: it is merely the exercise
> of basic property rights.  This is biased, inflammatory language which
> indicates, once again, a lack of even rudimentary knowledge on this issue.
>
> ---Rsk
> From: Nancy McGough <nm-reverse-spam-filter () ii deflexion com>
>
>
>
>  > Date: Thu, 04 Sep 2003 14:29:35 -0700
>  > From: Ed Gerck <egerck () nma com>
>  > Subject: we are all blacklisted
>  >
>  > The lesson here is that current systems for automatically
>  > handling large volumes of spam and virus, as well as
>  > "black-hole" lists, are backfiring.
>
> I think the main lesson that I've learned is that most people,
> including unfortunately people who produce filtering software, do
> not understand that the From:, To:, and Cc: headers do not
> matter. What matters is the SMTP envelope. A couple things we can
> do right now to help the situation are:
>
> * Never reject (bounce back to sender) a message after it leaves
>    the SMTP world.
>
> * Insist that mail-hosting providers inject headers that state
>    the original envelope sender and original envelope recipient.
>    This will help users to filter their messages appropriately.
>
> * Teach users that it is too late to "bounce back" a message by
>    the time they deal with it with their filters. Bouncing
>    (rejecting) can only be done correctly during the SMTP dialog.
>
> From: Rich Kulawiec <rsk () gsp org>
>
> On Thu, Sep 04, 2003 at 08:59:57PM -0400, Dave Farber wrote:
>  > >Anyone who owns a well-known email address is probably being
>  > >blacklisted right now by automatic spam- and virus-handling agents
>  > >worldwide, frantically trying to deal with SoBig and other culprits that
>  > >send emails from faked addresses.
>
> This is extremely unlikely.  Content-sensitive filters ignore the putative
> sender (since both spammers and virus/worms frequently forge it) and focus
> on the content.  E.g. AV programs look for signatures characteristic of
> known malware; anti-spam programs such as SpamAssassin look for evidence
> of spam by correlating against known spam samples.
>
> Neither "blacklists" any user addresses because there's nothing to gain
> by doing so.  (Though a few user addresses may be useful in accruing
> evidence: e.g. mail from big () boss com is probably an early SoBig variant,
> mail from CacheFlowServer@<anything> is probably a hijacked CacheFlow
> server being used to send spam, etc.  But there are only a handful of
> isolated examples such as this, and they're insignificant when compared
> to the millions of forged addressess -- some of which exist, some of
> which don't.)
>
> Non-content-sensitive filters ignore the originating address because
> it's usually of no value whatsoever -- see below.
>
>  > >Anyone whose address "looks good" is a preferential target. The lesson
>  > >here is that current systems for automatically handling large volumes of
>  > >spam and virus, as well as "black-hole" lists, are backfiring.
>
> I see absolutely no evidence that any such systems are "backfiring".
> Among other things, this isn't how DNSBLs [1] (aka "black-hole lists")
> work: they completely IGNORE the putative sender and focus solely on
> the originating IP address.  This in turn is one of the reasons that
> DNSBLs are highly effective -- they rely on information that's much
> harder to forge [2] than the sender.
>
> ---Rsk
>
> [1] A DNSBL is a DNS-based Block List.  All modern mail transport
> agents (MTAs) such as sendmail, postfix, etc. can use these.  The
> way they work is that when an incoming connection to the MTA shows up,
> the MTA queries one or more DNSBLs, using the originating IP address
> of the connection.  Various codes are returned (depending on the DNSBL)
> which indicate "okay", "spam source", "open relay", "open proxy", etc.
> The MTA then decides what to do with this information: block the message,
> drop the connection, tag the message, whatever.  DNSBLs acquire their
> information by various means, including automated testing of connecting
> MTAs, manual testing, spamtraps, and other means.  There are now several
> hundred of them in operation, nearly all run by volunteers.  See
> http://www.spamfaq.net/ or http://www.openrbl.org/ for more info.
>
> [2] But not impossible: some spammers have recently started forging
> documents claiming ownership of IP blocks abandoned during the dot-com
> flameout.  They then convince ISPs to route traffic for them, and use
> these "zombie networks" to spam (of course).  The countermeasure for
> this is to simply list the entire zombie network in the appropriate
> DNSBLs and deny all traffic from it.  See http://zombie.dnsbl.sorbs.net/
> for one such DNSBL.
>
> -------------------------------------
> You are subscribed as BobIP () Bobf Frankston com
> To manage your subscription, go to
>   http://v2.listbox.com/member/?listname=ip
>
> Archives at: http://www.interesting-people.org/archives/interesting-people/

-------------------------------------
You are subscribed as interesting-people () lists elistx com
To manage your subscription, go to
 http://v2.listbox.com/member/?listname=ip

Archives at: http://www.interesting-people.org/archives/interesting-people/