Re: [NSE] Two new scripts url-snarf, http-auth-finder

[Duarte Silva <duarte.silva () serializing me>]

On Sunday 29 January 2012 16:54:09 Patrik Karlsson wrote:
> On Sun, Jan 29, 2012 at 4:34 PM, Duarte Silva
>
> <duarte.silva () serializing me>wrote:
> > On Sunday 29 January 2012 16:22:07 Patrik Karlsson wrote:
> > > On Sun, Jan 29, 2012 at 3:27 PM, Duarte Silva
> > >
> > > <duarte.silva () serializing me>wrote:
> > > > On Sunday 29 January 2012 09:26:34 Patrik Karlsson wrote:
> > > > > Hi all,
> > > > >
> > > > > I've committed two new scripts:
> > > > > url-snarf: Sniffs the network for urls in HTTP traffic and
> > > > > dumps
> > > > > them
> > > > > together with their originating IP
> > > > > http-auth-finder: Spiders a site and detects web pages
> > > > > requiring
> > > > > HTTP- or form-based authentication.
> > > > >
> > > > > Comments and feedback is welcome.
> > > > >
> > > > > Cheers,
> > > > > //Patrik
> > > >
> > > > Hi Patrik,
> > > >
> > > > some notes for the url-snarf:
> > > > - it's missing the interface argument documentation and in the
> > > > example usage,
> > >
> > > it should appear as obligatory;
> > >
> > > > - in line 31, identation problems (/me being picky);
> > > >
> > > > For the http-auth-finder:
> > > > - identation in line 59 (/me being picky again :P );
> > > >
> > > > Wondering how http-auth script could take advantage of
> > > > "nmap.registry.auth_urls". Maybe if it was a post rule script?
> > > >
> > > > Regards,
> > > > Duarte Silva
> > >
> > > Thanks I just committed a version that addresses all those issues.
> > > In regards to the nmap.registry.auth_urls, the http-auth script
> > > would
> > > simply need a dependencies line containing the http-auth-finder
> > > script.
> > > This way the http-auth scripts doesn't run until the
> > > http-auth-finder has finished running.
> > >
> > > If someone want's to try to implement the necessary changes in
> > > http-auth, to leverage the auth_urls registry entry, please let me
> > > know.
> > >
> > > Cheers,
> > > Patrik
> >
> > I was tinking more in the lines of, after everything executes, post rule
> > would
> > be activated, and the script would check every URL found.
> > This would allow other scripts to be added that could fill in the
> > "nmap.registry.auth_urls" variable without having to mess with
> > dependencies.
> >
> > I don't see much of a problem in using the dependencies, but still, I'm
> > wondering if it wouldn't be best to add the postrule, so that http-auth
> > could
> > take advantage of any script adding auth_urls without having to mess
> > with
> > the
> > dependencies. Thoughs?
> >
> > Regards,
> > Duarte
>
> I'm not sure I see the problem/benefit? I mean the code change in the
> http-auth script would be more or less the same, ie. iterating over the
> contents of the registry entry and check authentication type. You would
> need to make sure to have both scripts on the command line, as we don't
> have forced dependencies, in both cases. The only difference would be
> adding a line with a dependency or adding a postrule right?
>
> I guess that the upside of the postrule is that you don't need to update
> the dependency line in http-auth if we decide to add another script making
> use of the registry entry auth_urls. However, I don't see this as a big
> benefit/problem.

Agreed.

> One thing we loose with the postrule is the grouping per port and host,
> which we get with the portrule. Allthough, we kind of already loose that if
> the spider is allowed to crawl outside of it's host and/or domain. Anyway,
> I still vote for implementing this with a dependency and consider binding
> the discovered URLs to a host and port entry in the registry in a better
> way than I've currently implemented.

True, they should be bind'ed to a host/port pair, since the auth_urls is 
global, an execution of the http-auth script against multiple hosts may fire 
authentication checks in url's found in previously checked hosts.

By the way, will we won't concurrency problems because of using the auth_urls 
with multiple hosts/script scans will we?

Best regards,
Duarte

> Cheers,
> Patrik

Attachment: smime.p7s
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/