Re: a few usability problems and how to scan very fast a large network

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 09 Mar 2010 22:17:23 +0100
Farkas Levente <lfarkas () lfarkas org> wrote:

> hi,
> i've got a few question.
> in short:
> 1. what's the fastest way to discover all available ip address in a 
> large (eg. class B network) if we're on a fast (at least 100Mb) LAN?

The best way would be to look at your ARP tables.  With Nmap though,
something like this should be pretty comprehensive:

nmap -v -d -v -sP -PE -PP -PM
- -PS21,22,23,80,135,139,443,445,1024,1025,1026,3389 -PA
21,22,23,80,135,139,443,445,1024,1025,1026,3389 -T5
- --min-hostgroup 2048 --min-parallelism 256 -oA results 10.10.0.0/16

> 2. is there any way to filter output based on the scan specification?

See below.

> 3. is there any usable output format?

All of them but for different purposes.

> in a longer version (start from the end):
> 3. the current formats are not really useful for automatic
> processing, since:


> - normal output is not very easy to parse.

Agreed.

> - grepable output has less info then xml and even less then normal 
> output, so in most case it's not enough.

Generally, yes.

> - xml output contains a lots of good for nothing tags. eg. if i scan
> a B subnet then the the resulted xml which contains ALL host tag ie.
> 65536 tag. why? even if we have a few dozens of used ip address we've
> to load and parse a rather large tree which is time and memory
> consuming proccess and totally redundant. this file is about 7MB !!!
> while read it while process and and why use memory for this.

Well if you tell Nmap to scan an IP and it doesn't get anything it
should record the null result.  It may not be useful for your purpose
but you can use Xquery/Xpath to filter that.

Also, 7MB is really nothing.  I've produced XML output from Nmap in the
2+ GB range for a single large scan.

> 2. to continue 3. suppose i like to see only those host which
> contains only those host which has open http port. if i add -pT:80
> then i will get the answer (open or closed) but probably don't really
> like to know those host which has closed http port. so suppose i only
> like to get an xml ouput for hosts which has an open http port i like
> to get a very short xml file and not a 7MB file.

Again, more information is always better, you can just ignore
information that isn't pertinent.

Also, if you're going for such targeted information as in your example
above:

$ sudo nmap -v -d -n -Pn -p 80 -T5 --min-parallelism 256 --min-hostgroup 2048 -oG - 192.168.0.0/16 2>/dev/null | awk 
'/^Host:.*Ports: 80\/open/ {print $2}'


> 1. i need to discover in a large network (B class or larger) all ip 
> cameras and find it's type. suppose if i've all ip address which have 
> open http port (all ip camera have open http port) then i can find
> it's type. so i 'only need' to find all ip address with open http
> port. how can i do that in the fastest way? currently i find/use:
> ----------------------------------
> nmap -n -PR -T5 -pT:80 --min-parallelism 1024 --max-parallelism 4096 
> 192.168.0.0/16

You really want to avoid TCP connections here.  Instead of using T just
- -sS and then -p 80

Also, -PR won't really work outside of your VLAN.


> ----------------------------------
> which require to rise ulimint -n and running time is 14 seconds.
> ----------------------------------
> nmap -n -PR -T5 -pT:80 --min-parallelism 512 --max-parallelism 1024 
> 192.168.0.0/16
> ----------------------------------
> 23 seconds and without parallelism parameter:
> ----------------------------------
> nmap -n -PR -T5 -pT:80 192.168.0.0/16
> ----------------------------------
> 969 seconds (!), from nmap's manual:
> "By default, Nmap calculates an ever-changing ideal parallelism based
> on network performance."
> but the last example show that it's not working very well.
> and i still not run my ip camera type recognition routine:-(

Okay lets try:

$ time sudo nmap -n -PN -p 80 -T5 --min-parallelism 4096 --min-hostgroup 8192 132.239.0.0/16 -oG - 2>/dev/null | awk 
'/Host:.*open/ {print $2}' | wc -l
2949

real    0m15.170s
user    0m5.262s
sys     0m2.121s

> now the problem is that exacqVision's ip server can scan the same (b 
> class) network in about 7 seconds and always find all ip cameras
> (both the windows and linux version of the server) with the correct
> camera type. so it seems they can know something much better the
> nmap:-( or is there any way to (command line) which i should have to
> use? so

A pcap for a /24 would be nice.  I'm guessing some sort of UDP based
fire and forget trick specific to the camera.

> - what's the fastest way to discover all available ip address in a
> large lan?

Available has lots of definitions but it sounds to me like any IP used
by a machine.  ARP tables are the only sure way to do that.

Try something like:

nmap -v -d -v -sP -PE -PP -PM
- -PS21,22,23,80,135,139,443,445,1024,1025,1026,3389 -PA
21,22,23,80,135,139,443,445,1024,1025,1026,3389 -T5
- --min-hostgroup 2048 --min-parallelism 256 -oA results 10.10.0.0/16

> or even better:
> - what's the fastest way to discover all available ip address which
> has open http port in a large lan?

See my above commands for flooding SYNs on port 80.


Brandon

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)

iEYEARECAAYFAkuWwZwACgkQqaGPzAsl94LB9ACfWpvY0swyfWes2z2WlQggn7Qz
b/cAn3R6ZqyW/wPrNrH8Jc0P6esVUGgz
=m2AM
-----END PGP SIGNATURE-----
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/