Mike Godwin <mnemonic () eff org> on Rimm for Hot Wired part 2 of 3

[David Farber <farber () central cis upenn edu>]

hours), I couldn't ethically approve of legal footnotes without seeing the
text of the article they were footnotes *to*. I pointed this out to Rimm
and suggested that, if he were to send me the full article, I might be
able to find the time to review the footnotes for any obvious mistakes.


Rimm told me he'd get back to me on that. But he never did. And the next
time I heard about the Rimm study was early in the week of when Philip
Elmer-DeWitt of Time called me early the week of the 19th for comment on
the Rimm study and the conclusions Rimm, who by now had received his
bachelor's degree, had reached. Among these conclusions, Philip told me,
were that tastes for online porn were becoming more "extreme," that adult
BBSs were using Usenet to market their wares, that sysops had discovered
that the more "violent" the language of a description the more popular an
image was, and that Amateur Action BBS, whose Milpitas, CA, sysops had
recently been successfully prosecuted in Memphis, Tennessee, was "the
market leader" of online porn.


It was clear from the questions Philip was asking that Time was going to
treat the Rimm study as a major story -- perhaps even a cover story. And
this insight was Part One of what I'd later think of as Philip's Triple
Whammy. Given what I already knew about Rimm's research, I was appalled
that Time would publicize it -- I immediately tried to warn Philip of the
methodological and other problems I saw with the study. He told me that
study was going to be published in an article in the Georgetown Law
Journal, that Time had an exclusive, and that he (that is, Philip
Elmer-DeWitt) found Rimm's methodology convincing. I couldn't believe we
were talking about the same study. Philip found it easy to dismiss my
caveats -- after all, I hadn't seen the study.


So I asked to see it -- I promised Philip that, if he showed it to me, I
wouldn't "leak" it, but instead would use it to frame more detailed and
substantive criticisms (or, perhaps, be forced to admit that the
methodology and conclusions were convincing after all). That was when
Philip hit me with the Second Whammy -- thanks to an arrangement with the
law journal and/or with Rimm (Philip was vague about this), *no one*
outside of the editors of Time and the law journal would get to see the
study before the Time story appeared and the law-journal issue was
published. I was stunned -- if there were questions about the study's
reliability (and I still had every reason to believe there were), the
arrangement Philip told me about practically *guaranteed* that those
questions wouldn't be fully considered by Time's editors. Especially since
Philip had already convinced himself that the doubts I tried to raise
weren't serious ones. I knew Philip to be by Time editorial management --
I'd even heard rumors of an upcoming promotion -- so I was certain that,
if Philip vouched for the reliability of the study, his superiors would
take his word for it.


So at this point I made two suggestions: First, I referred him to Donna
Hoffman, a Vanderbilt University professor I knew from the WELL. I knew
Hoffman and her husband Tom Novak to be among the most knowledgeable
people in the world when it came to questions of surveying Net usage or of
modeling marketing strategies in this new medium. I assumed that Hoffman
and Novak would raise the same methodological questions I had, plus some
I'd no doubt overlooked, and perhaps that would convince Philip to look
again at the reliability of the Rimm study.


My second suggestion was for Philip to contact whoever it was who was
insisting on nondisclosure of the article and ask them to grant me
permission to see it for comment, with the proviso that I'd agree not to
leak it in any way. This came to nothing -- when I reminded Philip about
it the following week, he professed not to remember that I'd ever proposed
this arrangement.


And although Philip did have one of Time's field reporters interview
Hoffman, he never spoke to her himself. He did read the "file" from the
reporter's interview, though. We know this because he later argued on the
WELL that the intensity of Hoffman's language in commenting on the Rimm
study methodology (she knew about it from the abstract and -- mirabile
dictu! -- from her own prior correspondence with Rimm, who'd solicited her
advice and support months before) made her an unreliable source. After
all, how could she be so critical *when she hadn't seen the study*? And,
of course, she was barred from seeing it by the arrangement among Time,
the law journal, and Rimm.


The more I thought about the study's imminent publication, the more
troubled I was by the secrecy and the lack of critical review. That's when
it occurred to me to consider how odd it was that an article by an EE
major, purporting to be a *marketing* study, was appearing in a *law
review*. Although Philip took this to be an index of the study's likely
reliability, I knew something that, at least at first, he did not --
namely, that law reviews are unlike most other scholarly journals in that
they're edited not by professors or professional editors but by
*third-year law students*. And while I have the highest regard for the
ability of student law-review editors at a school like the Georgetown
University Law Center, I knew it was highly unlikely that the editorial
staff had the expertise to question the claims and arguments that Rimm
would be making about his computer-mediated research into the "information
superhighway." Suddenly the legal footnotes took on a new significance --
they were the thin entering wedge that qualified Rimm's article as a fit
piece for a law review.


It all came together for me then. If Rimm had set out to publish an
article about online porn in a way that *legitimized* his article yet
*escaped* the kind of critical review the piece would have to undergo if
published in a scholarly journal of computer-science, engineering,
marketing, psychology, or communications, *what better venue than a law
journal?* And a law-journal article would have an added advantage -- it
would be read by law professors, lawyers, and legally trained policymakers
and taken seriously. It would automatically be catapulted into the center
of the policy debate surrounding online censorship and freedom of speech.


I tried to point this out to Philip when he called me back for a second
interview, but he clearly wasn't terribly interested in hearing it -- he
grunted obligingly, but moved to the questions he really wanted to ask me,
about the net.censorship legislation pending in Congress and about what I
thought the effect of the publication of the study and its appearance in
Time would be. "It will be a disaster," I told him. "It won't matter if
you try to balance your presentation of the study with the questions
people have about its methods and reliability. It'll be used to stoke the
fires of the Great Internet Sex Panic." He noted my comments, then ended
the conversation.


As the days counted down to publication of the next issue of Time, I
indulged in hopeful thoughts. Philip had a great track record as a
reporter on cyber-issues -- for all that even the most balanced story
would be, in my view, "a disaster," I could understand how Philip had
convinced himself of the importance of the story, and, as a once and
future journalist myself, I respected his commitment to tell a story even
if the facts might generate the wrong kind of reaction among policymakers
or the public. Not once in all my discussions with him had I ever
suggested that he not do the story. And when it came to the critical issue
of balance, I fancied that I could trust in his professionalism. Indeed,
when rumors of the upcoming Time story had surfaced, and some WELL users
were ready to castigate Philip for writing it, I posted the following
one-line message on Sunday morning, June 25: "Let's hold off criticizing
Time until we see what the story looks like."


But all this hope left me wide open for what would turn out to be Part III
of the Triple Whammy.


Here's what I posted on Monday, when I had had a chance to read the piece
as it appeared in Time:




--------------------


media.1029.86: Avant Garde A Clue (mnemonic)  Mon 26 Jun 95 14:39




Philip's story is an utter disaster, and it will damage the debate about
this issue because we will have to spend lots of time correcting
misunderstandings that are directly attributable to the story.


For example, when Philip tells us what the Carnegie Mellon researchers
discovered, he begins his list with this:


'THERE'S AN AWFUL LOT OF PORN ONLINE. In an 18-month study, the team
surveyed 917,410 sexually explicit pictures, descriptions, short stories
and film clips. On those Usenet newsgroups where digitized images are
stored, 83.5 percent of the pictures were pornographic.'


Who but the most informed among us will not come away with the impression
that the CMU study involved a survey of 917,410 items *on Usenet*? (Guess
what -- it didn't.)


And he concludes the list with this;


"IT IS NOT JUST NAKED WOMEN. Perhaps because hard-core sex pictures are so
widely available elsewhere, the adult BBS market seems to be driven
largely by a demand for images that can't be found in the average magazine
rack: pedophilia (nude photos of children), hebephilia (youths) and what
the researchers call paraphilia--a grab bag of "deviant" material that
includes images of bondage, sadomasochism, urination, defecation, and sex
acts with a barnyard full of animals.'


Problem is, this isn't the typical range of content you find in Usenet
newsgroups, or on commercial services, or even on most BBSs. Instead, this
is the range of content you find on the specialized subclass of commercial
BBSs that focus on pornography.


Just to make things worse, Philip refers to the Internet in the next two
grafs (and not at all to commercial porn BBSs).


This is an incredibly muddled abortion of a story, despite Philip's
attempts to introduce balance. The *packaging* of the story -- a cover
with an innocent child at a keyboard, the paintings of  men fucking a
computer or being pulled into one -- is deeply  sensationalistic.


And the profound problems with the study's methodology go undiscussed.
Sure, we have a guy pointing the possibility of a "gaper" phenomenon,
which tells us something about how to interpret the results of a
correctly conducted survey. But not a hint of how methodologically flawed
the study is, or about how the people doing the study were rank amateurs,
or about how the legal footnotes were spiced with citations from anti-porn
zealots like Catharine MacKinnon and Bruce Taylor.


The Time story aims at legitimizing the study as raising important issues.
What it does instead is raise serious questions about whether the lure of
an exclusive eclipsed Time's professional judgment.




----------------------------




And in the course of the next few days, I questioned Philip pointedly
about the writing and editorial decisions he'd made in the Cyberporn cover
story -- decisions that both maximized the extent which the story
exacerbated the Great Internet Sex Panic and actually *obscured* critical
facts about the study. Philip occasionally responded with glib,
superficial answers, which enraged me. It was as if he were deliberately
ignoring the magnitude of what he'd done.


Now it wasn't my researcher buttons that were being pushed -- it was my
journalism buttons. Philip had written the story in such a way that, in
effect, he would be deceiving great numbers of his readers. With a copy of
the study in hand (finally!) I began to savage Philip in the media
conference on the WELL:


----------------------




 media.1029.102: Avant Garde A Clue (mnemonic)  Mon 26 Jun 95 20:27




Philip writes:


"Well, it *was* a graph about adult BBSs, wasn't it?"


Philip, this is the most infuriatingly disingenuous answer I can imagine
your making. *Did you not read my criticisms above?* You conflate Usenet,
Internet, and BBSs so readily in your listing of the study's conclusions,
that the *nuance* that a *particular* graph is about particular subniche
of commercial BBSs--and not the Internet--*is certain to be lost to any
reader who is not knowledgeable about the medium, and to many that are.*


I pointed out *already* that the next two graphs *following* your
hebephilia-bestiality graph mention the Internet, not BBSs. Are you simply
*oblivious* to the meaning communicated by that juxtaposition? Perhaps you
are, since *you get confused yourself*. In one of those next two graphs in
which you mention the Internet, you say: "The Internet, of course, is more
than a place to find pictures of people having sex with dogs."


Just one problem -- you haven't said even once, prior to that, that the
study shows that bestiality images are common on the Internet.


I'm going to put the lie to your disingenuous claim. Let's just look at
how you juxtapose these three paragraphs:


'IT IS NOT JUST NAKED WOMEN. Perhaps because hard-core sex pictures are so
widely available elsewhere, the adult BBS market seems to be driven
largely by a demand for images that can't be found in the average magazine
rack: pedophilia (nude photos of children), hebephilia (youths) and what
the researchers call paraphilia--a grab bag of "deviant" material that
includes images of bondage, sadomasochism, urination, defecation, and sex
acts with a barnyard full of animals.


'The appearance of material like this on a public network accessible to
men, women and children around the world raises issues too important to
ignore--or to oversimplify. Parents have legitimate concerns about what
their kids are being exposed to and, conversely, what those children might
miss if their access to the Internet were cut off. Lawmakers must balance
public safety with their obligation to preserve essential civil liberties.
Men and women have to come to terms with what draws them to such images.
And computer programmers have to come up with more enlightened ways to
give users control over a network that is, by design, largely out of
control.


'The Internet, of course, is more than a place to find pictures of people
having sex with dogs....'




Now, Philip, please tell me, with a straight face, that you think  a Time
reader who is not knowledgeable about this medium will *not* draw the
conclusion that the Rimm study shows that the *Internet* is the place
where all those images can be found.


Explain the goddamned "sex with dogs" link, Philip.


I have no patience with this dishonest bullshit.




media.1029.103: Avant Garde A Clue (mnemonic)  Mon 26 Jun 95 20:44




Philip also writes, incredibly:


"Also, you seem to have bought into the as yet unproven assertions about
"profound problems with the study's methodology." So far, the chief
criticism that's been leveled against it here is that it was headed by an
undergraduate. I'm waiting for something more specific. And meaningful."


What makes this an incredibly dishonest statement is that, when Philip
called me for commentary about the study, *I begged for a copy of it, so I
could review the methodology.* Not only that, but I gave you contact
information for Donna Hoffman, prof () well com, who is indisputably
competent to critique the methodology, and you *didn't give her a copy
either*.


Still, since Rimm discusses the methodology in the abstract (which I've
posted in this topic), both Donna and I were able to make several
methodological critiques of what the authors *said* they were doing.


<expletive-ridden commentary from Godwin deleted>


You've totally lost it, Philip. The statement that you haven't heard
anything except a complaint about Rimm's age, undergraduate status, and
inexperience is flatly a lie.


But what's worse is the lie that you tell by implication. Please quote the
passage in your story where you *mention* that Rimm, "the study's
principal investigator," is an undergraduate EE major with no former
experience in studying or applying the statistical methodology used in
conducting surveys.


What? You omitted to mention it? Now, why did you do that, Philip? Could
it be because you wanted to give the impression that Rimm is far more
authoritative than, in fact, he is? Because that would improve the cachet
of your exclusive?


Don't even bother talking to me any more. After the immense dishonesty of