Re: New whitehouse.gov robots.txt file

[David Farber <dave () farber net>]

Begin forwarded message:

From: Dave CROCKER <dhc2 () dcrocker net>
Date: January 21, 2009 10:47:09 AM EST
To: dave () farber net
Cc: ip <ip () v2 listbox com>, Joseph Lorenzo Hall <joehall () gmail com>,  
Declan McCullagh <declan.mccullagh () cnet com>
Subject: Re: [IP] New whitehouse.gov robots.txt file
Reply-To: dcrocker () bbiw net



> Begin forwarded message:
> Here's another fascinating sign of increased transparency in the new
> administration:
> The whitehouse.gov robots.txt file -- a file that specifies what areas
> of a web site that web spiders may crawl[1] -- has gone from 2400
> lines to just two lines:


Declan McCullagh has just posted an article that explains the error in  
this overly simplistic metric:

Much ado about Whitehouse.gov's new openness
<http://news.cnet.com/8301-13578_3-10146802-38.html?part=rss&tag=feed&subj=News-PoliticsandLaw 
>

d/
--

 Dave Crocker
 Brandenburg InternetWorking
 bbiw.net




-------------------------------------------
Archives: https://www.listbox.com/member/archive/247/=now
RSS Feed: https://www.listbox.com/member/archive/rss/247/
Powered by Listbox: http://www.listbox.com