PaulDotCom mailing list archives

new version of CeWL out


From: dninja at gmail.com (Robin Wood)
Date: Mon, 18 Jan 2010 11:40:50 +0000

Hi
I've just released a new version of CeWL, my Custom WordList
generator. This version now scans pages for javascript location
redirects and adds them to the spider list. This helps with a problem
Josh Wright found with a site where its index page was just a redirect
to the main framework.

I also found a bug in the ruby spider gem and have a short write up on
how to fix that, basically it only hits urls where in the href the url
is surrounded by double quotes so it ignores single quotes.

Unfortunately I didn't get chance to get the non-ascii character stuff
sorted out for this version, I know a few people asked me for it.
Hopefully next time unless anyone knows how to get ruby to handle
non-ascii characters correctly and is willing to help out.

http://www.digininja.org/projects/cewl.php

Robin


Current thread: