Re: HTTP scanners? (summary, long)

[BlueBoar () THIEVCO COM (Blue Boar)]

Got a ton of responses on this.  Rather than try and be selective about
the replies or put through a bunch of single messages, I decided
to summarize.  If anyone who posted is really insulted by this
(perhaps I cut your new .sig) let me know off-list.

                                        BB

From: Arturo Busleiman <buanzox () usa net>
well, there is a small program called cgichk, and here I show you the
output produced after running it against my box at home (yes, it's and
old distribution, I know, but I have modified lots of things, and
sincerely, It would be a pain in the ass to update it):

buanzo:~$ cgichk localhost

                 [CKS & Fdisk]'s CGI Checker

         [ Press any key to check out the httpd version...... ]

HTTP/1.1 200 OK
Date: Tue, 25 Jan 2000 13:08:53 GMT
Server: Apache/1.2.4 S.u.S.E./5.1
Last-Modified: Fri, 23 Oct 1998 11:40:55 GMT
ETag: "11741-44c-36306b47"
Content-Length: 1100
Accept-Ranges: bytes
Connection: close
Content-Type: text/html

         [ Press any key to search 4 CGI stuff...... ]
Searching for phf : Found !! ;)
Searching for Count.cgi : Found !! ;)
Searching for test-cgi : Found !! ;)
[etc]

the cgi-scan is disastrous (as you can see :)
If you (#define you 'the guy who started this thread') want the source,
I'm sure you can find it anywhere (cgichk.c :), but you can ask me, ok?

bye!

From: "WHiTe VaMPiRe" <whitevampire () mindless com>

    A little shell script that ATTRITION wrote may be of help, with a bit
 of modification.

ros is available at: http://www.attrition.org/tools/

From: phazer <phazer () talocan dhs org>

Hi,

I wrote a program that does what you need a while ago but i didn't intend
to publish it and I haven't looked at it in a while..

if you want to check it out i put up the source at
http://talocan.dhs.org/wscan.c

From: <admin () superdups com>

http://packetstorm.securify.com/UNIX/scanners/httpscan-v200.c
http://packetstorm.securify.com/exploits/ADM/z0ne.c

httpscan reads in a file of servers to scan...so you could z0ne a
domain and run httpscan on that file.

sometimes the http header doesnt really tell you enough so you could use
nmap -p 80 -O to find out the os type..  for a in `cat blah` ; do nmap -p
80 -O -o $a.log $a; done

From: "Jay D. Dyson" <jdyson () techreports jpl nasa gov>

        There is such a thing.  I wrote a script suite that performs this
task for JPL/NASA once it was learned that there were no official
statistics on how many webservers ran on our networks and the type of OS
on which they were run.  My script suite does the following things:

        1.      Find httpd's where running on ports 80, 81, 8000, 8080,
                8100 and 8888.  Scanning for all open ports and then
                tickling each to see if they proved to be httpd proved
                time-consuming and yielded no appreciable results.

        2.      Connects to those ports that answer and uses HEAD to
                acquire the httpd version running.

        3.      Uses nmap (v2.12) -O to fingerprint the system OS.

        4.      Logs the above data (system IP, answering port, httpd
                make and ver, system OS) in a carat-delimited file
                which can be readily manipulated in PERL and, where
                necessary, imported into MS Excel for managerial
                purposes.  (Most management types don't grok either
                PERL or UNIX, so this affords them the ability to
                work with the data without asking me to sort the data
                in different ways.)

        The script suite functions pretty nicely and performed quite well
in a scan of nearly 20,000 systems.

From: "Seth Georgion" <SysAdmin () sassproductions com>

Why don't you look for a scanner under tools at Security Focus. There =
are tons of scanners that accept a list of ports 80, 81, 443 and scan =
whatever Subnet you specify.

From: <rpc () inetarena com>

Seth, Rory, et al,
  This is trivial to do. Finding the version of a webserver is as easy as
issuing a "HEAD / HTTP/1.0" and parsing the "Server" field. Other than the
popular cgi scanners, the only scanners i've seen that were designed to
just do this have been home grown. It's pretty easy in perl.

From: hypoclear - lUSt - (Linux Users Strike Today) <hypoclear () jungle net>

I would say to take an existing "banner scanner" and modify
it to look for the banners for IIS, Apache, etc.  If you
want to look for what port the webserver is running on, you
could do a portscan of the entire computer, then see which
banners match up where, however this would be VERY noisy
(unless you did things like NMAP scans).  The best would be
to default to the ports 80 or 8080, and possibly 443 for
https.  Most of the time webservers arent run on other
ports.  PERL would be useful for a project like this...

From: Stefan Aeschbacher <stefan () aeschbacher com>

I don't know of such a utility but with a small sh-script (and maybe
nmap)
you can get the information you desire. Just do the following:

1) Get a list of hosts and ports.
   Use: host -l domain.xxx|cut -f4 '-d ' to get a nice list of all
   servers in this domain.
   If you want to search on every open port, make a scan of your list
   with nmap, else just use port 80.
2) write a small script like this one:
   whathttpd.sh:
   #!/bin/sh
   # create unique process sync file
   WHATHTTPD=/tmp/whathttpd-$RANDOM-$$
   export WHATHTTPD
   touch $WHATHTTPD

   # print name of checked server
   echo $*:

   # connect, request and wait until finished
   # for generic port replace $* 80 with $*
   (echo "GET / HTTP/1.0\n\n";while [ -f $WHATHTTPD ];do sleep 1;done) |
   (telnet $* 80 2> /dev/null| grep 'Server:';rm $WHATHTTPD)
3) feed the script with you list

This process could be automated by another script, so writing such a
a tool is a matter of minutes.

From: Peter Drapich <docent () union pl>

I wrote such util - it scans hosts given in a textfile and saves replies in
separate files containing returned server types, replies received and
returned
errors. Very informative:)) I wrote also similar util for smtp but its not
so
usable...

From: "Marc" <marc () eeye com>

You should check out Grinder version 2.0. It lets you set what HTTP ports
you want to scan and also the usual URL grinding. So just hit up
80,8080,8000 etc... for whatever URL you want to find.
ftp://ftp.technotronic.com/rhino9-products/Grinder2.zip

It would only take a little bit of time to tweak up the code
(ftp://ftp.technotronic.com/rhino9-products/grinder_source.zip) to port
scan
each IP for common HTTP ports and then just grind those etc...

From: root <root@localhost.localdomain>

Try http://www.netcraft.com. It does something similar to that.

From: rain forest puppy <rfp () wiretrip net>

You could always use nmap to find the ports, and whisker to report on
them.  ;)

Although you'd probably want a custom script (and obviously one that does
not scan for CGIs...)

I suppose I could produce an example script to demonstrate this, if anyone
wants.

From: Sozni <sozni () usa net>

Don't forget about good 'ol netcat
(http://packetstorm.securify.com/UNIX/netcat/)

I create file named c:\head that contains this:

HEAD / HTTP/1.0

Then you can run this command at a prompt:
  nc -v -w 3 www.wisesolutions.com 80 3128 8000 8080 < c:\head

Or if you are looking for non-standard ports:
  nc -v -w 3 www.wisesolutions.com 1-1024 3128 8000 8080 < c:\head

That will return server headers if there is an http server.  To scan the =
whole
network will take a bit of work, but if you do it often, you could create=
 a
batch file, pipe in addresses using a tool like xargs or just write a scr=
ipt
to scan an ip range.

From: "Bacano" <bacano () esoterica pt>

ftp://ftp.technotronic.com/rhino9-products/grinder_source.zip <= double
click =)

... but, in the other hand, ask for horizon his 0day eheheh

From: Raymond Medeiros <medeiros () ureach com>

You could write a simple TCP Connect Scanner that would open
every port looking for open ports.  When it finds an open port
you could have it throw out a GET and see if it acts like a web
server.  Then have it report the ip and port number.  Shouldn't
be that tough.