Re: foreign HTTP requests

[ddoc () MIA CZ (Daniel Docekal)]

> They send you cookies too .....
> Maybe a problem for privacy, no ?

I have made report to secure () microsoft com (repeated), now they have finally
reacted to it, but i am afrtaid they take it as something obscure and think
that i am insane or bugging them for nothing. For me it IS security
(privacy) problem with possible serious implications. It is not that big
problem for our webs since we have changes 404 scripts to just log these
request in different database.

> Any idea on how this can happen ?

Well, ideas are always present - imho it is bug in O/S resolver which keeps
somehow and sometime sending request to "previous" visited web site - what
bugs me is that this is NOT particular to some browser nor some platform.
Since we are getings THOUSANDS of these request a day (we run web with some
million of hits a day) i have tried to look for pattern in UserAgent - and
there is none. This bug is in Netscape/microsoft as well as in
Unix/Windows....

See current list (data from yesterday to today, first column is number of
entries, 2nd is useragent)

1264        Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)
953         Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)
344         HttpAgent 1.0 (unix)
157         Mozilla/4.0 (compatible; MSIE 5.01; Windows 98)
148         Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; CT2)
132         Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; DigExt)
88          Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)
65
63          Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; CT1)
51          Mozilla/3.0 (compatible; NetMind-Minder/4.01)
38          eCatch/3.0
34          Mozilla/2.0 (compatible; NEWT ActiveX; Win32)
33          Mozilla/4.0 (compatible; BorderManager 3.0)
29          Mozilla/3.0 (Win95; I; HTTPClient 1.0)
27          Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; {World Online})
26          Mozilla/4.0 (compatible; MSIE 5.0; Windows 98)
25          Mozilla/4.5 [en] (Win95; I)
25          Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; Computer
CD)
23          Mozilla/4.72 [en] (Win98; I)
21          Mozilla/4.0 (compatible; MSIE 4.01; MSIECrawler; Windows 95)
19          Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)
18          Mozilla/3.0 (compatible; NetMind-Minder/4.0)
17          Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)
15          Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt;
VOLIE_5_CZ_VOLNY)
14          Mozilla/4.08 [en] (WinNT; I ;Nav)
14          Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; Seznam)
13          Mozilla/2.0 (compatible; MSIE 4.0; Windows 98)
13          Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; EuroTel)
12          Mozilla/4.0 (compatible; MSIE 5.01; MSN 2.5; Windows 98)
11          Teleport Pro/1.29
10          Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; DigExt)
10          ExtractorPro
10          Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; VOLIE_5_CZ_VOLNY;
DigExt)
9           Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; DigExt; Seznam)
9           Mozilla/2.0 (compatible; PlanetWeb/1.011 Golden; SEGA Saturn;
TV; 640,480)
8           Mozilla/4.05 [en] (X11; I; OSF1 V4.0 alpha)
8           Mozilla/4.61 [en] (Win95; I)
7           Mozilla/4.6 [en] (Win98; I)
7           Mozilla/4.04 [en] (WinNT; U)
6           Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)
6           Mozilla/4.51 [cs] (Win98; U)
5           Mozilla/4.51 [en] (Win95; I)
5           Mozilla/3.01 (Compatible; Links2Go Similarity Engine)
5           Mozilla/3.0 (Win16; I)
5           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; Freei
Networks)
5           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt;
Superonline)
5           Mozilla/4.0 (compatible; MSIE 5.0; MSN 2.5; Windows 98;
CNETHomeBuild03171999)
5           Mozilla/2.0 compatible; Check&Get 1.2 (Windows NT)
4           Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
4           Mozilla/3.Mozilla/2.01 (Win95; I)
4           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; ZDNetSL; DigExt)
4           Mozilla/3.01 (compatible; netbox/3.5 R88; Linux 2.2)
3           Mozilla/4.0 (compatible; MSIE 5.0; AOL 4.0; Windows 98; DigExt)
3           Googlebot/2.0 beta (googlebot(at)googlebot.com)
3           Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; Windows User
Magazine)
3           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; Seznam)
3           Mozilla/2.0 (compatible; MSIE 3.01; Windows 95)
3           Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; Highway 194
Internet Vertriebs Ges.m.b.H)
2           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; KITV4
Wanadoo)
2           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; AltaVista
1.01.01; DigExt)
2           Mozilla/4.0 (compatible; MSIE 5.0; Windows 95)
2           Mozilla/4.0 (compatible; MSIE 4.01; Windows 98; DigExt)
2           Mozilla/4.5 [en] (Win95; I)(compatible; Powermarks/3.0; Windows
95/NT4)
2           Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; Seznam)
2           ia_archiver
2           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt;
VOLIE_5_CZ_VOLNY_I2000)
2           Mozilla/1.1N (Windows; I; 16bit)
2           Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)
2           Mozilla/4.51 [it] (Win98; I)
2           Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; DigExt;
VOLIE_5_CZ_VOLNY)
2           Offline Explorer/1.3
2           Mozilla/4.05 [cz] (Win95; I)
2           Mozilla/4.7 [en] (Win98; I)
2           Mozilla/4.7 [en] (Win95; I)
2           Mozilla/2.0 (compatible; MSIE 5.0; Windows 98)
2           Mozilla/2.0 (compatible; HTTPClient 1.0; Windows 95)
1           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; MSNCOM)
1           Mozilla/4.51 [en] (X11; I; SunOS 5.7 sun4u)
1           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; sunrise4)
1           Mozilla/4.0 (compatible; MSIE 4.01; Windows 98; WISH)
1           Mozilla/4.06 [en] (Win98; I)
1           Mozilla/4.0 [en] (Win95; I)
1           Mozilla/4.06 [en] (Direct Hit Grabber)
1           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; CZCOM)
1           Mozilla/3.04 (Win95; I)
1           Mozilla/4.0 (compatible; MSIE 5.0; Mac_PowerPC)
1           Mozilla/4.6 [de] (WinNT; I)
1           Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90)
1           Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; CT1;
MSIECrawler)
1           DaviesBot 1.7 (Licensed to WholeWeb.net, Inc.)
1           Mozilla/4.05 [en] (Win16; I)
1           Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; {World Online};
NET M@NAGER V2.0 - WWW.VINN.COM.AU)
1           Mozilla/1.1 (compatible; MSPIE 2.0; Windows CE)
1           Mozilla/4.51 [en] (Win98; I)
1           Mozilla/3.01 (Win16; I)
1           Mozilla/4.0 (compatible; MSIE 5.0; Win32)
1           Mozilla/2.0 (compatible; MSIE 3.02; Windows 3.1)
1           Mozilla/4.04 [en] (WinNT; I ;Nav)
1           Mozilla/4.51 [en] (X11; I; Linux 2.2.13 i686; Nav)
1           Mozilla/4.0 (compatible; MSIE 5.0; MSN 2.5; Windows 98)
1           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; OptusIE5)
1           Mozilla/4.04 [en] (Win95; U)
1           Tango/c1.2 c1.0 Win32 Alis/6
1           Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; QXW03306;
QXW03302; DigExt; QXW0331t)
1           Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; {World Online};
DigExt)

Daniel