Secure Coding mailing list archives
Re: Vulnerability Auditing Checklist
From: "Steven M. Christey" <coley () mitre org>
Date: Tue, 04 May 2004 14:22:42 +0100
Maybe Steven have a newer version available?
Unfortunately, I haven't been able to do a major overhaul, or to refine the categories.
Would be nice with some illustrations to each vulnerability listed.
Below is an updated version. There are some more sub-categories, and now most categories have some example vulnerabilities. It still needs a lot of work, but maybe it will be useful. I haven't had a chance to read it closely, but McGraw and Hoglund's new book "Exploiting Software" defines a number of specific attack patterns. There's also the OWASP WebApp Pen Test Checklist. Both of these are a good move forward in formalizing some aspects of auditing and, by extension, vulnerability research. - Steve ================================================================ Vulnerability Auditing Checklist ================================================================ Version: 0.0000004 Modified: 2004/05/03 (Change Log at end) Created: 2002/12/04 Disclaimer: This is a DRAFT document. The list of categories is incomplete. In addition, some categories overlap, and some terms are wholly invented or ill-defined. It has not been compared with other sources. This document has been publicly posted to facilitate discussion of code review/testing procedures, and to spur progress in vulnerability classification. Many of these categories include CVE identifiers (CVE-yyyy-nnnn or CAN-yyyy-nnnn) for specific vulnerabilities that demonstrate the given category. The identifiers can be accessed from the search form at http://cve.mitre.org/cve Notes on the Checklist ---------------------- The categories, as presented, sometimes mix the underlying programming error ("the vulnerability") with the attack vector and/or the consequence/impact of the issue. This is a reflection of the current state of affairs in vulnerability/attack research. Significant work is required to clearly delineate the boundaries between these closely related concepts. Some bug types are probably "under-studied" by researchers. That is, there may be many latent issues that remain undiscovered because researchers have not been paying close attention to them. These types are appropriately noted, based on the author's experience. General Unexpected or Malformed Input Problems ---------------------------------------------- 1. Buffer Overflows Terminology Note: many issues that are now called "buffer overflows" are substantively different than the "classic" overflow, including entirely different bug types that rely on overflow exploit techniques, such as integer signedness errors and format string bugs. This imprecise terminology can make it difficult to determine which variant is being reported. 1a. Boundary end violation ("classic overflow") Note: from the standpoint of programming errors, stack-based and heap-based overflows may not differ significantly (closer study is needed). Examples: many, such as CVE-2000-1094 1b. Boundary beginning violation ("buffer underflow" ?) Note: term may have been used for multiple issue types; concept seems understudied Ref: VULN-DEV:20040110 Buffer UNDERFLOWS: What do you know about it? Examples: BUGTRAQ:20020911 Buffer over/underflows in ssldump prior to 0.9b3 1c. Array index modification (also called "index overflow" or "out-of-bounds array index" or "index-out-of-range") Note: involves accessing memory outside an allocated buffer, and exploit uses buffer overflow techniques, but this can be exploited without "large inputs" Note: can overlap integer signedness errors, parameter/arg tampering Examples: CAN-2003-0072, CAN-2004-0093, CAN-2001-1009, CAN-2001-1036, CAN-2002-1066, CAN-1999-0798, CAN-2003-0721, CAN-2004-0093 1d. Length parameter manipulation ("length tampering") Definition: the attacker can manipulate the length parameter associated with an input, and provide an input of a different length. Note: probably overlaps other categories Examples: CAN-2001-1186, CVE-2001-0191, CAN-2003-0429, CVE-2000-0655, others 1e. Off-by-one error Note: not always a buffer overflow Examples: CAN-2001-1391, CVE-2002-0083, CVE-2002-0653, CAN-2002-0844, CAN-2004-0342 (this is an interesting example that might not be an off-by-one) 1f. Other length calculation error Note: this is a broad category. Some examples include: (1) simple math errors, (2) incorrectly updating parallel counters, (3) not accounting for size differences when "transforming" one input to another format (e.g. URL canonicalization or other transformation that can generate a result that's larger than the original input, i.e. "expansion") Examples: likely many, with variants such as those described above. However, this level of detail is rarely available. 2. Format strings Note: format string issues seem to be heavily under-studied for languages other than C. Memory or disk consumption, control flow or variable alteration, and data corruption may result from format string exploitation in applications written in other languages such as Perl, PHP, Python, etc. Ref: Format String Vulnerabilities (Tim Newsham) Examples: many, including CVE-2000-0573, CAN-2000-0574, CAN-2004-0354, CAN-2004-0232, CAN-2004-0159, CAN-2004-0104 3. Syntax/grammar violation Note: the general problem of "malformed input" is under-studied from the standpoint of underlying programming errors. Most efforts have been in developing attack methods, which rarely suggest the nature of the underlying error. Attack-related research in this type of "malformed input" is scattered but ongoing, e.g. see fuzzers, suite-based testing (PROTOS style), and fault injection. The effect of these is often a denial of service, although other impacts may be under-studied. However, the specific underlying cause is rarely diagnosed by the researcher. This is especially problematic when researchers report that "a number of random inputs were provided, which led to a crash." 3a. "Empty" or blank input Note: some of "crash by port scan" bugs are probably due to this, but lack of diagnosis makes it difficult. Examples: CAN-2002-0422 (overlaps information leak) 3b. Missing argument Examples: CAN-2004-0276, CAN-2002-1488, CAN-2002-1169, CVE-2000-0521, CVE-2001-0590, CAN-2002-1236, CAN-2003-0239, CAN-2003-0477, CAN-2003-0422 3c. Extra argument Examples: N/A 3d. Repeated argument Examples: N/A 3e. Missing/repeated/extra separator or delimiter Examples: CAN-2002-1362, CVE-2000-0116, CAN-2001-1157 3f. Wrong data type Definition: the application does not properly handle input data that is of the wrong type, e.g. it expects a digit (0-9) but is provided with a letter (A-Z). Examples: CVE-1999-1156 3g. Incomplete input Note: overlaps incomplete resource release Examples: CAN-2002-1532, CAN-2003-0195 3h. Missing/misplaced special characters (delimiters/etc.) Examples: N/A 3i. Unknown/unrecognizable argument/command/whatever Examples: CVE-2001-0650 4. Special character mismanagement Note: some bug types overlap this category, e.g. SQL injection and LDAP injection. Note: some of these types of special chars have been observed at one point or another, but it's difficult to construct the proper query to find suitable examples. In an attempt to be complete about what kinds of "special characters" exist, some types may have been added to this list without any publicly reported vulnerability for those types. Note: some buffer overflow vulnerabilities rely on mismanagement of special characters 4a. Shell metacharacters Note: same as item 11 Examples: see item 11 4b. Delimiter between fields Examples: CAN-2003-0307 4c. Delimiter between values Examples: CVE-2000-0076, CAN-2000-0293 4d. Delimiter between records Examples: N/A 4e. CRLF injection (line delimiter) Ref: CRLF Injection (Ulf Harnhammer) Note: probably under-studied. Examples: CAN-2002-1575, CAN-2003-0336, CVE-2000-0610, CAN-2001-0902 4f. Section delimiter Note: overlaps CRLF injection (e.g. CRLF between MIME headers and content) 4g. End-of-input delimiter (e.g. "." in mail message data) Examples: CVE-2000-0319, CVE-2000-0320, CAN-2001-0996 4h. Input terminator Note: overlaps null character injection Examples: N/A 4i. Quoting character Examples: N/A 4j. Escape/meta/control character Examples: CAN-2000-0476, CAN-2003-0083 (overlaps interaction error), CAN-2002-0985 (overlaps argument injection), CVE-2002-0542, CAN-2003-0021, CAN-2003-0022, CAN-2003-0023, CAN-2000-0476, CAN-2001-0523 4k. General separator char Examples: N/A 4l. Comment char Examples: CAN-2002-0001 4m. Macro symbol or other char for substitution Examples: CAN-2002-0770 4n. Variable name leader/terminator (e.g. "$" for env. variable) Examples: N/A 4o. Wildcard or "completion" character Examples: CAN-2002-0433 (overlaps equivalent filename), CAN-2002-1010, CVE-2000-0587 4p. Disallowed character Definition: does not properly handle a character that is specifically disallowed by specification or other convention. Note: overlaps wrong data type, probably others Examples: CAN-1999-1265 5. Dependent Field/Value Inconsistency (e.g.: a "length" field for a buffer does not reflect the actual length of the buffer; or, two fields have values that do not make sense when combined) Note: partially overlaps length parameter manipulation (1d) Examples: N/A (difficult to search for examples of this type) 6. Null dereference Note: typically this is the result of another vuln (e.g. failed error handling, malformed inputs); however, most vulnerability reports only list the null dereference and not the underlying trigger. Note: some attacks that use long input values can trigger null dereferences, which are then mis-diagnosed by the researcher as buffer overflows. Examples: CAN-2004-0365, CAN-2003-1013, CAN-2003-1000, CAN-2004-0389 (overlaps malformed inputs), CAN-2004-0119 File/Directory Processing ------------------------- 7. Directory traversal and filename equivalents Terminology note: some people use "directory traversal" only to refer to the injection of ".." and equivalent sequences whose specific meaning is to traverse directories. Other variants like "absolute pathname" and "drive letter" have the *effect* of directory traversal, but some people may not call it such, since it doesn't involve ".." or equivalent. Note: filename equivalents can be regarded as a type of canonicalization error. Note: this item should be split into multiple sub-categories, but for the sake of consistency with the numbering convention in earlier versions of this checklist, they were combined. Note: Incomplete diagnosis or reporting of vulnerabilities can make it difficult to know which variant is affected. For example, a researcher might say that "..\" is vulnerable, but not test "../" which may also be vulnerable. Note: any combination of the items below can provide its own variant, e.g. "//../" is not listed (CAN-2004-0325). Note: most of these issues are probably under-studied. 7a. ../filename Examples: many 7b. /../filename Examples: many 7c. /absolute/pathname/here Examples: CAN-2000-0614, CVE-2001-0265, CVE-2000-0778, CAN-2000-0845, CVE-2000-0915, CVE-2000-1196, CAN-2001-1269, CVE-2001-0677, CAN-2003-0753 7d. /directory/../filename Examples: CAN-2001-1152, many others 7e. directory/../../filename Examples: many, including CAN-2002-0298 7f. ..\filename ("dot dot backslash") Examples: many, including CAN-2002-0661, CVE-2002-0946, CAN-2002-1042, CAN-2002-1209 7g. \..\filename Examples: N/A 7h. \absolute\pathname\here ("absolute path") Examples: many, including CVE-1999-1263, CAN-2003-0753, CAN-2002-1344, CAN-2002-1525, CAN-2000-0614, 7i. \directory\..\filename Examples: N/A 7j. directory\..\..\filename Examples: CVE-2002-0160 7k. C:dirname or C: ("drive letter") Examples: CAN-2001-0687, CAN-2001-0933, CAN-2002-0466, CAN-2002-1483 7l. ... (triple dot) Examples: CVE-2001-0963, CVE-2001-0615, CAN-2001-0419, CAN-2003-0425, CVE-2001-0467, CAN-2002-0325, CAN-2003-0313, CAN-2002-0288, CAN-2002-0261, CAN-2001-1335, CVE-2001-1193, CAN-2001-1131, CAN-2002-0784, CAN-2002-0708 7m. .... (or other multiple-dots) Examples: CVE-2000-0240, CVE-2001-0615, CAN-1999-1082, CAN-2001-0491 7n. \\UNC\share\name\ Examples: CAN-2001-0687 7o. filename.ext. (trailing dot) Examples: CAN-2001-1386 7p. filename.ext[SPACE] (trailing space) Examples: N/A 7q. dirname. (trailing dot) Examples: CAN-2004-0061 7r. dirname[SPACE] (trailing space) Examples: CVE-2001-0054 7s. dirname/ (trailing slash) Examples: CAN-2002-0253 (overlaps infoleak), CAN-2001-0446 7t. dirname/filename.ext/ (trailing slash) Examples: CAN-2001-0893, CAN-2001-0892 7u. //multiple/leading/slash ("multiple leading slash") Examples: CVE-2000-1050, CAN-2002-1483, CVE-1999-1456 7v. /multiple//internal/slash ("multiple internal slash") Examples: N/A 7w. /multiple/trailing/slash// ("multiple trailing slash") Examples: N/A 7x. \double\\windows\\separator Examples: N/A 7y. /./ Examples: possibly CAN-1999-1083 (could be a cleansing error) 7z. dirname/fakechild/../realchild/filename Examples: CAN-2001-1152 7aa. dirname.... Example: BUGTRAQ:20040205 Apache + Resin Reveals JSP Source Code ... 7ab. filename... Example: BUGTRAQ:20040205 Apache + Resin Reveals JSP Source Code ... 8. Link Following Note: symlink following bugs are the combination of multiple elements: file or directory permissions, filename predictability, etc. Note: UNIX hard links and Windows hard/soft links are probably vastly under-studied. Note: Windows soft links have been exploited remotely. Note: these can overlap filename equivalents. 8a. UNIX symbolic link (symlink) following Examples: many, including CVE-1999-1386, CVE-2000-0972, CVE-2000-1178, CAN-2004-0217 8b. UNIX hard link Examples: CAN-2002-0793, CAN-2003-0578, CVE-1999-0783 8c. Windows .LNK Examples: CVE-2000-0342, CAN-2001-1042, CVE-2001-1043 8d. Windows hard link Examples: CAN-2002-0725, CAN-2003-0844 9. Windows 8.3 filenames Note: these are probably under-studied Examples: CAN-2001-0795, but there are others 10. "Virtual" files 10a. Windows MS-DOS device names Examples: CAN-2002-0106, CAN-2002-0200, CAN-2002-1052, CVE-2001-0493, CVE-2001-0558, CVE-2000-0168, CAN-2001-0492 10b. Windows ::DATA alternate data stream Examples: CVE-1999-0278, CVE-2000-0927 (note: there may be others with different attack vectors and impacts) 10c. Apple ".DS_Store" Examples: BUGTRAQ:20010910 More security problems in Apache on Mac OS X Process/Command Execution ------------------------- 11. Shell metacharacters Examples: many, including CVE-1999-0066, CVE-2001-1246, CVE-2002-0061, CAN-2003-0041 12. Malicious search path execution (search path can be modified by untrusted user to point to malicious program, e.g. UNIX PATH environment variable) Note: this doesn't just apply to Unix. Examples: CAN-2002-1576, CAN-2000-1128, CAN-1999-1461, CVE-1999-1318 13. Program/command argument modification or argument injection Examples: CAN-2004-0121, CAN-2003-0907, CVE-1999-0113 Canonicalization Errors ----------------------- 14. Encodings Note: partially overlaps directory traversal/filename equivalents Note: many other types of encodings should be listed here 14a. URL encoding (hex encoding) - %2e (encoded dot "." - overlap directory traversal) - Examples: CAN-2004-0072 - %5c (encoded backslash "\" - overlap directory traversal) - Examples: CAN-2004-0072 - %0a (overlap CRLF) - Examples: CAN-2002-1575 - %20 (encoded space " " - overlap filename equivalent) - Examples: CAN-2004-0280, CAN-2003-0424, CAN-2001-0693 - %00 (encoded null - overlap filename equivalent, factor in directory traversal) - Examples: CVE-2000-0671, CAN-2004-0189, CAN-2002-1291, CVE-2002-1031, CAN-2001-1140 - + (may be converted to a space when decoding the query string) - Examples: CAN-2002-1451 14b. Unicode Examples: CVE-2000-0884, CAN-2001-0709, CAN-2001-0669 (overlaps interaction error) 15. Multiple separators or other characters Note: overlaps the current items in the directory traversal / filename equivalents section. Examples: N/A 16. Case sensitivity Note: these are probably under-studied in Windows environments, e.g. case sensitivity in filenames (see filename equivalents). Examples: CVE-2000-0497, CVE-2000-0498, CAN-2001-0766, CAN-2001-0795, CAN-2001-1238, CAN-2003-0411 17. Validate-Before-Canonicalize Definition: a program "validates" data before it is canonicalized. This can allow attackers to "insert" invalid data by causing it to be produced by canonicalization. Note: this is probably a general category, and we may see more of it as programmers begin to take defensive steps, but take those steps in the wrong order. Examples: CAN-2002-0433, CAN-2003-0332, CVE-2002-0802 18. Validate-Before-Cleanse Definition: a program "validates" data before it has been cleansed. This can allow attackers to make data look "clean" at first, but the cleansing step produces "dirty" output. Note: this is probably a general category, and we may see more of it as programmers begin to take defensive steps, but take those steps in the wrong order. Note: currently seen mostly in directory traversal Note: this category is probably under-studied. Examples: CAN-2002-0934, CAN-2003-0282, possibly CAN-2003-0417 Leaks ----- 19. Information Leak Note: there seem to be many different types of problems that lead to information leaks, whose severity can range widely depending on the type of information that is leaked. In addition, information leaks are often the result of other bugs. 19a. Sensitive information not cleared after use Examples: CAN-2001-0984 19b. Sensitive memory not cleared due to compiler removal Examples: BUGTRAQ:20021105 When scrubbing secrets in memory doesn't work 19c. Command-line arguments visible to other processes Examples: N/A 19d. Environment variables visible to other processes Examples: N/A 19e. State information leak due to inconsistent responses (e.g. user name enumeration: valid username/wrong pass generates "incorrect password," but invalid username generates "incorrect user") Note: can overlap errors related to escalated privileges Examples: CAN-2004-0294, CAN-2004-0243, CAN-2002-0514, CAN-2002-0515, CAN-2001-1387 19f. State information leak due to timing discrepancies (e.g. a "successful" operation takes more time than an unsuccessful one) Examples: CAN-2003-0078, CAN-2000-1117, CAN-2003-0637 19g. Incomplete removal of temporary resources (e.g. files) Examples: CVE-2001-0465 19h. Application-controlled diagnostic or error messages Examples: N/A 19i. Uncontrolled, external diagnostic or error messages (e.g. the programming language leaks information on an error that happens in the application) Note: PHP applications are often targeted for having this issue when the PHP interpreter generates the error outside of the application's control. Examples: N/A 19j. Design-intended or configuration-intended leak (information is intended for publication, but sensitive) Examples: CAN-2003-1038, CAN-2004-0033 (and many other PHP apps that support phpinfo) 19k. Sensitive information not cleared before reuse or initialization Examples: CAN-2003-0001, CAN-2003-0291, CAN-2004-0177, CVE-2001-1074, CAN-2002-0339 19l. Sensitive information leaked via "stale" pointer Examples: CAN-2004-0080 20. Resource leaks 20a. UNIX file descriptor leak Examples: CVE-1999-0083, CAN-2003-0740, others Multiple Operation/Action Errors -------------------------------- 21. Duplicate operation 21a. Double-free Examples: CVE-2002-0004, CVE-2000-0550, CAN-2002-0847, CVE-2002-0059 21b. Double-encoding / double-decoding Note: overlaps canonicalization. Probably under-studied. Examples: CVE-2001-0333 22. Improper handler deployment (dispatch error) Note: may be the same as item 28. 23. Inability to handle out-of-order actions (state machine violations). Examples: possibly CAN-2000-0647 and/or CAN-2000-0648 (diagnosis incomplete; could be null dereferences or similar issues) 24. Race Condition (non-file link) 24a. Signal handler race condition Note: probably under-studied. Examples: CVE-2001-1349 24b. Other TOCTOU (time-of-change time-of-use) Examples: N/A 25. Deadlock Note: this is under-studied relative to vulnerability research. This item is more of an effect that is triggere by an underlying bug. Examples: CAN-2001-1400 Configuration Errors -------------------- Note: configuration errors are vastly under-studied, especially in terms of classification. They can be more complex than vulnerabilities (which are often discrete and easily separable). In addition, configuration overlaps with the general area of "policy" which can have elements that are not always considered to be relevant to security. 26. Permissions, ACLs, and ownership 26a. Bad default or inherited permissions (read, write, execute) Examples: many 26b. Bad program-assigned permissions (read, write, execute) Examples: many, such as CVE-2002-0265, CAN-2003-0876 26c. Ownership of critical resource not verified Examples: many 26d. Does not properly manage failures due to insufficient permissions Note: probably under-studied. Examples: CAN-2004-0148 27. Default configuration enables insecure feature 27a. Default password Note: default passwords should probably be distinguished from hard-coded passwords. Examples: many 27b. Default, non-essential service or component Examples: many 27c. Network-based admin capability accessible to arbitrary hosts by default Examples: many Error Condition Identification/Management Errors ------------------------------------------------ 28. Handler dispatch error Note: may be the same as item 22. Note: may be the result of another vulnerability 28a. Improper handler deployment (the wrong "handler" is assigned to process an input, e.g. calling a servlet to reveal source code of a .JSP file, or automatically "determines" type even if contradictory to an explicitly specified type) Examples: CVE-2002-0025, CVE-2000-0682, CVE-2000-0778, CAN-2000-1052, CVE-2001-0004, CVE-2001-0126 28b. Missing handler (handler not available or implemented) Examples: CAN-2002-0614, CVE-2001-0330 28c. Dangerous handler not cleared/disabled during sensitive operations 29. Insufficient logging of security-critical events Note: should be distinguished from cases when logging does not coccur as the result of other vulnerabilities (e.g. those that cause a crash) Examples: CAN-2000-0118, CVE-2000-0542, CVE-2000-0937, CVE-2001-0056, CAN-2001-0471, CAN-2001-0978 30. Incomplete error detection (product does not properly detect or check for security-critical error conditions) GUI Errors ---------- 31. Insufficient user warning of "unsafe" actions Examples: CVE-2001-0072, CAN-2001-0919, CAN-2000-1138 (overlap error handling?) 32. Interface inconsistency Definition: an API, GUI, or other interface behaves inconsistently with what operations are actually performed on the system, e.g. checking a security option does nothing, or user tells interface "restrict ALL" and it says "restrict SOME") Note: can be the result of another issue Examples: CVE-2000-0127, CAN-2001-1387, CVE-2000-0225, CVE-2000-1169, CVE-2001-0003, CVE-2001-0081, CAN-2001-0617, CVE-2001-0863, CVE-2001-0864, CVE-2001-0865, CVE-2001-0866, CVE-2001-0867, CVE-2001-0969, CAN-2001-1387 Product Management Errors ------------------------- Note: many of these categories cover specific phases of the software life cycle, which is often independent of the category of bug that is involved. However, some types of bugs seem to be inherent to certain phases of the life cycle, so at this early stage, these categories may be useful as a "catch-all" for certain issues. 33. Design limitations Note: this seems under-studied, especialy with respect to classification of design flaws. 33a. Incomplete specification 33b. Vague specification 33c. Support (or lack of support) for security-relevant options 34. Distribution Error 34a. Debugging code not omitted from production version Note: exposes vuln's in the debugging code itself; perhaps this is not a category Examples: CVE-1999-0095, CVE-2001-0528 34b. Product is shipped with insecure configuration that was used to facilitate internal testing. Examples: CAN-2003-0983 35. Patch Error 35a. Regression error - an old vulnerability is introduced into new code 35b. Incomplete vulnerability fix 36. Documentation Error 36a. Omission of security-critical information 36b. Error/typo causes user to introduce a vulnerability or risk 37. Developer-introduced back door / Trojan Horse Examples: CVE-1999-0145 38. Port Error Definition: A product is ported to a different environment (e.g. OS) and does not consider differences with the original environment - sometimes introducing vulnerabilities specific to the new environment. Examples: CAN-2002-0661 (overlaps directory traversal) 39. Interaction Error Two independent products work correctly and according to specification, but interact in ways that cause problems. Note: this category needs refinement. 39a. Interpretation conflict ("Conflicting Interpretation Error"): A interprets inputs differently than B. Note: this is generally found in proxies, firewalls, anti-virus software, and other "mid-stream" devices that allow or deny traffic based on how the client or server is expected to behave. Examples: classic Ptacek/Newsham IDS design flaws, CAN-2002-1121 (anti-virus), poison NULL byte between Perl and C 39b. Incomplete awareness: A is not aware of all types of input that B can take. Technology-Specific Problems ---------------------------- This is probably missing a number of issues in web technologies. 40. Cross-site scripting (XSS) Terminology Note: some people distinguish between XSS and "HTML injection." Terminology is imprecise for this category, which has a number of variants. Note: overlaps special char management Note: this category needs work Examples: many, such as CVE-2002-0074, CVE-2002-0938, CVE-2002-1053, CAN-2002-0530, CAN-2002-0840 41. Form field / parameter tampering Note: overlaps other categories; not just present in web apps. Examples: many, including CVE-2000-0758, CAN-2000-0554 (overlap informtion leak), CVE-2000-0926, CAN-2002-1352, CVE-2000-0253, CVE-2000-0254, CAN-2000-0101, CAN-2000-0102 42. SQL injection Note: overlaps special character mismanagement. Examples: many, including CAN-2004-0366, CAN-2004-0343, CAN-2003-0779, CAN-2003-0500, CAN-2003-0377 43. PHP-specific issues Note: these may overlap other categories, especially design Ref: A Study in Scarlet (Clowes) 43a. PHP remote code injection Examples: many, including CVE-2002-0953, CAN-2004-0285, CAN-2004-0030, CAN-2004-0068, CAN-2004-0127, CAN-2004-0128 43b. PHP untrusted external initialization of critical variables Note: can overlap PHP remote code injection, other types Examples: CVE-2000-0860, CAN-2002-0764, CAN-2001-0854, CAN-2001-1025, CAN-2003-0754 (overlaps authentication) 44. Perl null character injection (technically an interaction vulnerability, but important to mention specifically) Note: overlaps directory traversal / filename equivalents Other Errors ------------ 45. Initialization Error 45a. Insecure default initialization (e.g. variables or permissions) Note: overlaps other categories 45b. Untrusted/externally controlled initialization of trusted variables or values Note: overlaps other categories Examples: CVE-2000-0959, CVE-2001-0033, CAN-2001-0084, CVE-2001-0872 45c. Non-exit on failed initialization affecting security-critical resource (e.g. configuration file format error) 46. Resource exhaustion (memory, application-specific objects, general objects) Note: exhaustion is often the result of another bug. 46a. Memory leak Definition: software does not sufficiently track and release allocated memory when it is finished, which slowly consumes remaining memory. This is often triggered by improper handling of malformed data or unexpectedly interrupted sessions. Note: memory leaks are probably more often the result of an underlying bug, than an inherent bug themselves. Terminology Note: "memory leak" has sometimes been used to describe other kinds of issues, e.g. for information leaks in which the contents of memory are inadvertently leaked (CAN-2003-0400 is one such example of this terminology conflict). Examples: CAN-2004-0221 (overlaps other bug types), CAN-2004-0222, CVE-2001-0136, CAN-2001-0337, CVE-2002-0574, CAN-2004-0427 46b. Other incomplete resource release Definition: resource is not "released" for re-use or deletion, often as a result of an unusual error Note: overlaps memory leaks, asymmetric resource consumption, malformed input errors Examples: CVE-1999-1127, CVE-2001-0830, CVE-2002-0741, CAN-1999-1569, CAN-2002-1372 46c. Asymmetric resource consumption (aka "amplification") Definition: an "untrusted" entity can make a "trusted" entity consume more resources than necessary. Sometimes a factor in "flood" attacks, but other types of amplification exist. Note: There are probably several sub-types. Ref: Algorithmic Complexity Attacks (Crosby, Wallach) Examples: network amplification: CAN-2001-1244, CVE-2000-0041, CVE-1999-1379, CVE-1999-0513 algorithmic complexity: CAN-2003-0364, CAN-2003-0244, CAN-2002-1203, CAN-1999-1537, CVE-2000-1184; similar issues exist in cryptography CAN-2004-0002 identifies both subtypes. 46d. Insufficient resource pool Definition: the software's resource pool is not large enough to handle peak demand, which allows an attacker to prevent others from accessing the resource by using a (relatively) large number of requests for resources. Frequently the resource is a "flood" of connection or sessions. Note: "large" is relative to the size of the resource pool. See examples. Note: floods often cause a crash or other problem besides denial of the resource itself; these are likely examples of *other* vulnerabilities, not an insufficient resource pool. Note: overlaps other categories. Examples: CAN-2002-1063, CAN-2002-0782, CVE-2002-0406, CAN-2002-0234 47. Numeric conversion errors 47a. Integer Signedness Error (aka "signed integer" error) Note: overlaps array out-of-bounds Examples: CVE-2001-1279, CAN-2003-0619, CAN-2003-0972, CAN-2002-1420, CAN-2003-0166, CAN-2003-0721, CAN-2004-0094 47b. Integer overflow / underflow (value "resets" to maximum or minimum, often through incrementing values) Note: exploit sometimes uses buffer overflow attack Note: terminology may be inconsistent. Examples: overflows: CVE-2002-0391, CVE-2002-0639, CVE-2001-0144; possible underflows: CAN-2003-0761, CAN-2004-0219, CAN-2004-0184 47c. Network byte ordering mismanagement Examples: N/A 47d. Truncation error (happens when bits are truncated, e.g. due to casting or conversion between numeric types) Examples: N/A (but Halvar Flake mentioned some of these in his talk at CanSecWest 2004) 48. Authentication Error Note: this is a broad category, as there can be multiple errors that lead to authentication problems. More study is needed. Note: often, an authentication error is the result of a vulnerability. 48a. Authentication bypass by alternate path Definition: multiple "execution paths" exist for accessing resources that require authentication, but the authentication step is not placed in front of all paths. Seen in web applications that assume access to a particular CGI program can only be obtained through a "front" screen. But this problem is not just in web apps. Examples: CVE-2000-1179, CAN-1999-1454, CVE-2000-0944, many web applications 48b. Authentication bypass by alternate name Note: overlaps equivalent encodings, canonicalization Examples: CAN-2003-0317 48c. Dependency on user-controlled key data: user controls key data elements that are used for authentication, e.g. if a web application relies on a cookie "Authenticated=1" Examples: CVE-2002-0367 (DebPloit) 48d. Logic error Note: overlaps other categories; in these cases, an authentication bug is the *result* of a logic error. Examples: CAN-2003-0750 (conditional should have been an 'or' not an 'and') 48e. Partial password comparison Definition: the application uses the length of an input password to compare against the target password, which can allow an attacker to easily conduct brute force guessing. Examples: CAN-2002-1374, CVE-2000-0979, CAN-2001-0088 49. Unnecesarily large privilege window (app runs at higher privileges longer than it "has to") Note: difficult to distinguish this from failures to drop privileges. Examples: N/A 50. Capability operating at higher privilege than necessary without authentication Note: probably overlaps authentication Examples: N/A 51. Infinite loop Note: this is more the result of a programming error. Multiple sub-categories likely. More study is needed. Note: can overlap amplification or flooding Examples: CVE-2000-0620, CAN-2000-1203, CVE-2000-0738, CAN-2002-1355 (overlaps integer signedness) 52. Incomplete/missing security check for standardized algorithm/technique. Note: this is an implementation error, in which the algorithm/technique requires certain security-related behaviors that are not implemented, thus causing a vulnerability Note: overlaps cryptographic errors Examples: "Basic Constraints" browser cert issues (CAN-2002-0862, CAN-2002-0970, CAN-2002-1407), other issues (CVE-2000-0974 53. Cryptographic error Note: this category is incomplete and needs refinement, as there is some good documentation of cryptographic flaws. 53a. Stores sensitive data in plaintext (passwords, credit cards, etc.) Note: often overlaps bad permissions problems Examples: CVE-1999-0982, CVE-2000-0402, CAN-1999-1256, CAN-2002-0344, many others 53b. Does not use peer-reviewed cryptographic algorithms Note: this is a "best practice" more than a vulnerability Examples: N/A 53c. Does not perform all required cryptographic steps Note: overlaps incomplete/missing security check Examples: N/A 54. Insufficient Randomness 54a. Predictable from system or network state (time, process ID, etc.) Note: overlaps symlink attacks, cryptographic errors, information leaks Examples: CVE-2002-0389, CVE-2001-1141 54b. Insufficiently large space of random values (susceptible to brute force) Examples: CAN-2002-0583, CAN-2002-0903, CAN-2002-1107, CAN-2004-0230 (overlaps design limitation) 54c. Use of "known weak" randomness algorithms Note: probably a number of sub-types here Examples: CAN-2000-0916 54d. Does not initialize PRNG Note: overlaps predictable system state Examples: CVE-2002-0872 54e. Predictable from previous "random" values Examples: CAN-2002-1463 55. Miscellaneous remote code injection (inputs are fed directly into an interpreted language which is dynamically evaluated; other "classes" such as SQL injection are covered elsewhere) Examples: CVE-2000-0353, CAN-2002-0317, CAN-2001-1192 56. LDAP injection Note: overlaps SQL injection, special character mismanagement Note: probably under-studied Ref: Web Applications and LDAP Injection (SPI Dynamics) Examples: no public examples known yet, but probably frequently found by pen testing companies in custom apps Gaps in the Checklist --------------------- These items are known "gaps" relative to the above checklist, but they have not been studied well enough to place them into categories. This list is not complete relative to the known issues, but time constraints prevented a more complete list being provided. - Shatter style attacks Note: it is suspected that a number of publicized vulnerabilities that involve local privilege escalation on Windows systems may be related to Shatter attacks, but are not labeled as such. BUGTRAQ:20020806 White paper: Exploiting the Win32 API Examples: CAN-2002-0971, CAN-2002-1230, CAN-2003-0350, CAN-2003-0908 - web browser vulnerabilities, especially cross-domain and similar categories related to "sandboxing" Note: some browser bugs have a high complexity that makes them difficult to classify. These might be regarded as "technology-specific" except that they typically involve crossing security boundaries, and thus may have equivalents in other technologies that involve security models for multiple "domains." These bugs may stem from more basic programming errors, and thus may be an *effect* of those errors. Examples: many, such as CAN-2004-0380, CAN-2003-1026, CAN-2002-0284 - Windows named pipes Note: probably under-studied. Note: these sometimes play an important factor as an "alternate path" into a system, or an "equivalent filename." Examples: CVE-2002-0066, CAN-2003-0230, CAN-2003-0496 - physical access Note: problems that require physical access to exploit are probably under-studied, and may have certain categories that are unique to physical access. - Application-specific logic errors Note: this covers logic errors that are specific to a particular application, which can only be viewed as vulnerabilities within the context of that application's implicit or explicit security model. "Price tampering" in CGI applications has some overlap here, though that is not a solid example (see "form field" item 41) - "partial password comparison" is an example of a much larger category which involves the comparison or management of "smaller" items as if they were "bigger" items, or vice versa. Examples: CAN-2002-0499, CAN-2003-1025, CAN-2003-0412 - authorization errors - privilege errors - susceptibility to certain attacks (brute force, spoofing) - dotless IP addresses Examples: CVE-2001-0664, CAN-2001-0724 ========== Change Log ========== 0.0000001 - initial version, posted to secprog on 2002/12/05 0.0000002 - lost in the mists of time 0.0000003 - some dir. trans variants added, posted to Vuln-Dev on 2003/06/13 0.0000004 - extended some categories - added CVE examples - added notes for individual categories - added "Notes on the Checklist" - added "Gaps in the Checklist"
Current thread:
- Vulnerability Auditing Checklist Mads Rasmussen (May 03)
- <Possible follow-ups>
- Re: Vulnerability Auditing Checklist Steven M. Christey (May 04)
- Re: Vulnerability Auditing Checklist Alfonso Alba García (May 05)
- RE: Vulnerability Auditing Checklist Barney, Rickie (May 04)