Interesting People mailing list archives
IP: World Wide Web indexes: a study
From: Dave Farber <farber () central cis upenn edu>
Date: Tue, 31 Oct 1995 09:41:53 -0500
From: LEIGHTON () VAX2 Winona MSUS EDU I would like to announce the results of a study that I did in May and June of 1995. I compared the precision of four of the larger free World Wide Web indexing services: Infoseek (the service available for free), Lycos, Webcrawler and WWWWorm. For precision among the top ten hits (sites) returned, I used a two-way ANOVA to judge significance. I did this project for a graduate course in Computer Science. Although I did not get an outstanding grade on the project, I feel that there are real results and that they are important enough to post on the Internet. THE GIST OF THE RESULTS Cutting to the chase, my basic finding was that Lycos and the free part of Infoseek have about the same precision, with Lycos just a nose ahead. I was surprised and pleased with the precision of both. Webcrawler gave me surprisingly bad precision, surprising because it is currently supported by America Online. WWWWorm turned out to be a bit of a downer. My initial searching indicated that it was big league, but more extensive efforts proved that my impression was wrong. There are many smaller indexing services with it in this minor league: Jumpstart, Nikos, etc. I suspect that they do not enjoy enough institutional support for hardware to develop rich databases. These services are fairly abysmal, often retrieving nothing for a subject unrelated to computer science. WWWWorm was good enough that it usually retrieved at least one or two hits, which for my queries had high precision. As far as response time between Infoseek and Lycos, Infoseek was better, especially during the workday when traffic is up. However, Infoseek would block the free users if the paid traffic was too high, which it occasionally was. Also, at that time, the free Infoseek only returned the top ten hits, whereas now, it will provide many more. I have the complete report available on the Web at: http://www.winona.msus.edu/services-f/library-f/webind.htm Please comment if you have problems with the methods or conclusions. I will try to make constructive criticism available on the Web too. You may email me at: Leighton () vax2 winona msus edu H. Vernon Leighton Government Documents Librarian Winona State University Winona MN 55987 (507) 457-5148 Leighton () vax2 winona msus edu
Current thread:
- IP: World Wide Web indexes: a study Dave Farber (Oct 31)