Nmap Development mailing list archives
Re: Password Profiling and Password Mangling Libraries
From: George Chatzisofroniou <sophron () latthi com>
Date: Mon, 3 Jul 2017 17:25:40 +0300
Dan proposed to include all profiling code to unpwdb (instead of a separate pwdprofile library) [1] and I too believe it's better that way. I'm not sure I understand the "pwdprofile.parse_html" method. Parsing or decoding various formats is not part of the password profiler. It is the script's responsibility to parse or decode this data and then pass any interesting candidate phrases to profiler (e.g. with your "add_phrase" method). [1]: http://seclists.org/nmap-dev/2017/q1/48 George On Sun, Jul 2, 2017 at 12:22 AM, Wong Wai Tuck <wongwaituck () gmail com> wrote:
Hi list, I will be working on the password profiling library and password mangling library in the coming weeks. This is a follow up from the improvements and discussions first proposed by my mentor George [1] and the improvements suggested by Dan [2]. Below shows the design plan for the two libraries. pwdprofile.lua The aim of this library is to keep track of possible username/password candidates that are found from other scripts (e.g. http-title, http-passwd) so that they can be used in the later part (for bruteforcing or to export out to another program). pwdprofile.lua will also help to keep track of which are the information gathering scripts that should run first, so that they can simply be simply included into current brute scripts (e.g. ftp-brute). By default, all information gathering scripts will log the candidates to the library, and an optional argument will allow the user to specify if he wishes export the list out to a csv file. Script Args pwdprofile.export_file: the file to export data to, Default: nil pwdprofile.disable: disables pwdprofile, no new candidates will be added, Default false pwdprofile.custom_file: the file to import candidates from, Default nil pwdprofile.by_subnets: comma separated list of subnets that passwords will be aggregated by, rather than on a global basis Default nil pwdprofile.by_host: enables aggregation by host, rather than a global aggregation, Default false Script Constants: pwdprofile.STOP_WORDS_EN: a table of stopwords common in the English language pwdprofile.PWDPROFILE_SCRIPTS: a table of scripts that adds password candidates to pwdprofile Script methods: pwdprofile.add_word(host, keyword): saves a word verbatim to the password profile table. As new words are added, if the export option is specified, the word is also added to the file. pwdprofile.add_phrase(host, phrase, include_stop_words): parses a phrase for unique words, optionally including stop words like 'the' [3]. Calls add_word to add words. Default value for include_stop_words: false pwdprofile.parse_html(host, html, include_stop_words): parses the given html page (as a string) for unique words, optionally including stop words like 'the'. pwdprofile.get_candidates(host): returns the table containing the username/password candidates that apply to the current host, given the arguments, whether it is global (default), by subnet, or by host pwdmangle.lua The aim of this library is to provide an interface to stream possible username password candidates into the iterator in unpwdb. By default, no mangling is done so it streams whatever was originally in unpwdb. When rules are provided, it will mangle the passwords provided by unpwdb and pwdprofile, applying the rules given to generate additional words on the fly through lazy evaluation. While I have yet to write something like this before, I believe it can be done through a stateful class, where each brute script will call unpwdb to create a new instance of pwdmangle, and stream the password accordingly, so ideally the code for the brute scripts won't change, but changes will be made to unpwdb such that the next suggested password to use will be from pwdmangle. Script Arguments pwdmangle.rules: file that specifies the rules to apply. Rules can include our own (e.g. 1337 speak substitution) and those used by password crackers [4]. In the new domain specific language we specify the rules in, we should be able to combine lists, specify which particular list we are mangling (the unpwdb provided one, the pwdprofile, or the custom wordlists via their index in the comma separated arguments) or if we are mangling all lists globally. Default nil pwdmangle.wordlists: custom wordlist(s) provided by the user,comma separated; Default nil pwdmangle.ignore_default: ignores the default list given by unpwdb. Default: false pwdmangle.export: exports the passwordlist generated out to the file provided. Useful for debugging passwords Class Mangler(host, unpwdb_passwds, wordlists, rules): constructor that creates a new instance of Mangler based on the host, the password lists and the rules specified in the arguments. The default instance simply does what unpwdb does currently, and doesn't do any mangling. Otherwise, it creates a stateful instance which keeps track of what the next password should be based on what has been generated before and the rules given, and dynamically generate them. mangler.get_next_password() returns the next password candidate based on the current state of the object, or nil if there are no more passwords to iterate through. The original passwords (given by unpwdb and pwdprofile are prioritized first, followed by the rules applied). unpwdb Modifications will be made to unpwdb to get passwords from pwdmangle, passing in the original table so that if the user wants to apply the mangling strategies to the original password list they can do so as well, without breaking support for existing scripts users may have written. The dependencies will be as such brute-script ----depends on----> unpwdb ---> pwdmangle --->pwdprofile Scripts affected by this change discover/version/vuln/exploit scripts (adding useful information to pwdprofile) *-brute scripts (adding dependencies) unpwdb Do let me know if you have any feedback for the design plan for the two libraries! [1]: http://seclists.org/nmap-dev/2016/q2/46 [2]: http://seclists.org/nmap-dev/2017/q1/67 [3]: https://en.wikipedia.org/wiki/Stop_words [4]: https://hashcat.net/wiki/doku.php?id=rule_based_attack With Regards Wai Tuck _______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
_______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Password Profiling and Password Mangling Libraries Wong Wai Tuck (Jul 01)
- Re: Password Profiling and Password Mangling Libraries George Chatzisofroniou (Jul 03)
- Re: Password Profiling and Password Mangling Libraries Wong Wai Tuck (Jul 03)
- Re: Password Profiling and Password Mangling Libraries George Chatzisofroniou (Jul 05)
- Re: Password Profiling and Password Mangling Libraries Wong Wai Tuck (Jul 03)
- Re: Password Profiling and Password Mangling Libraries George Chatzisofroniou (Jul 03)