Re: [NSE] Two new scripts url-snarf, http-auth-finder

[Patrik Karlsson <patrik () cqure net>]

On Sun, Jan 29, 2012 at 4:34 PM, Duarte Silva
<duarte.silva () serializing me>wrote:

> On Sunday 29 January 2012 16:22:07 Patrik Karlsson wrote:
> > On Sun, Jan 29, 2012 at 3:27 PM, Duarte Silva
> >
> > <duarte.silva () serializing me>wrote:
> > > On Sunday 29 January 2012 09:26:34 Patrik Karlsson wrote:
> > > > Hi all,
> > > >
> > > > I've committed two new scripts:
> > > > url-snarf: Sniffs the network for urls in HTTP traffic and dumps
> > > > them
> > > > together with their originating IP
> > > > http-auth-finder: Spiders a site and detects web pages requiring
> > > > HTTP- or form-based authentication.
> > > >
> > > > Comments and feedback is welcome.
> > > >
> > > > Cheers,
> > > > //Patrik
> > >
> > > Hi Patrik,
> > >
> > > some notes for the url-snarf:
> > > - it's missing the interface argument documentation and in the example
> > > usage,
> >
> > it should appear as obligatory;
> >
> > > - in line 31, identation problems (/me being picky);
> > >
> > > For the http-auth-finder:
> > > - identation in line 59 (/me being picky again :P );
> > >
> > > Wondering how http-auth script could take advantage of
> > > "nmap.registry.auth_urls". Maybe if it was a post rule script?
> > >
> > > Regards,
> > > Duarte Silva
> >
> > Thanks I just committed a version that addresses all those issues.
> > In regards to the nmap.registry.auth_urls, the http-auth script would
> > simply need a dependencies line containing the http-auth-finder script.
> > This way the http-auth scripts doesn't run until the http-auth-finder has
> > finished running.
> >
> > If someone want's to try to implement the necessary changes in http-auth,
> > to leverage the auth_urls registry entry, please let me know.
> >
> > Cheers,
> > Patrik
>
> I was tinking more in the lines of, after everything executes, post rule
> would
> be activated, and the script would check every URL found.
> This would allow other scripts to be added that could fill in the
> "nmap.registry.auth_urls" variable without having to mess with
> dependencies.
>
> I don't see much of a problem in using the dependencies, but still, I'm
> wondering if it wouldn't be best to add the postrule, so that http-auth
> could
> take advantage of any script adding auth_urls without having to mess with
> the
> dependencies. Thoughs?
>
> Regards,
> Duarte


I'm not sure I see the problem/benefit? I mean the code change in the
http-auth script would be more or less the same, ie. iterating over the
contents of the registry entry and check authentication type. You would
need to make sure to have both scripts on the command line, as we don't
have forced dependencies, in both cases. The only difference would be
adding a line with a dependency or adding a postrule right?

I guess that the upside of the postrule is that you don't need to update
the dependency line in http-auth if we decide to add another script making
use of the registry entry auth_urls. However, I don't see this as a big
benefit/problem.

One thing we loose with the postrule is the grouping per port and host,
which we get with the portrule. Allthough, we kind of already loose that if
the spider is allowed to crawl outside of it's host and/or domain. Anyway,
I still vote for implementing this with a dependency and consider binding
the discovered URLs to a host and port entry in the registry in a better
way than I've currently implemented.

Cheers,
Patrik
-- 
Patrik Karlsson
http://www.cqure.net
http://twitter.com/nevdull77
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/