Secure Web Application Framework Manifesto

[rklists at gmail.com (Rohit Sethi)]

Paco, these are really great comments and will be really useful for us
in improving the doc prior to release as an OWASP project.

One aspect of the paper that I noticed you referenced a few times is
the set of requirements for authentication. I agree this isn't
normally the kind of thing a framework provides, but we've started to
see frameworks like Django offer authentication out of the box by
default (http://www.djangobook.com/en/2.0/chapter14/). Maybe we should
revisit whether or not to still include these since they are
application specific.

As for some of the specific concerns you have, let me attempt to respond

> > Some of the problems are broad (data confused as commands) and some are highly >>specific (getting down to the name 
> > of the researcher who discovered it).

If you don't mind I'd like to connect off-list with you and get
information about the specific problems are you are outlining in this
sentence. In particular we don't want to miss out on crediting the
right people.

> > Some seem like they mention technology that's cool without describing the problem that >>the technology addresses and 
> > why that technology is the right answer to the problem.

An example or two would really help here. Perhaps you mean
requirements like "Automatic Content Security Policy (CSP) Header
Generation" and how we don't go into detail about Cross Site
Scripting. Would a link to the OWASP article on XSS help? As for
describing why CSP mitigates against XSS, we feel that's adequately
documented in the CSP link we provided. Basically we don't want to
saturate the document with descriptions of problems that have already
be written about in detail, but maybe we didn't link to these.

> > Some seem like issues that a framework could offer (pluggable security) and others seem l>>ike they are operating 
> > system issues (security logs) while still others seem like they are >>application business logic (login forms and 
> > password requirements).

We are in now way saying that you must use a framework to solve the
problem, but rather that a framework CAN solve the problem. I'd be
interested to hear how security logs only apply to operating systems
and not to broad application behavior as defined in AppSensor.

> > Providing some set of criteria and/or features that indicate compliance with certain best >>practices is a fine 
> > idea. Should you spend your time enumerating flaws and attacks, >>though, or spend time enumerating correct 
> > behavior? I say this because the criteria in this >>document do both. In some cases they are prescriptive: "use 
> > cryptographic random >>numbers." In other cases, they are untestable negatives: "don't accept illegal characters."

I would suggest that we are always enumerating correct behavior.
"Don't accept illegal characters" is not an untestable negative; we
can verify that only a set of specific, legal characters (e.g. valid
UTF-8 byte sequences) are allowed by the framework and all others are
discarded or generate an exception. Maybe the point you are making is
that we should always word things in the affirmative? I.e. "Only
accept legal characters for a given encoding format". I think that's a
good idea


> > I think each of your criteria should have the same qualities that good software >>requirements have. They're often 
> > remembered using the mnemonic "SMART". Different >>people assign slightly different names, but I would suggest that, 
> > for your purposes, you >>consider: Specific, Measurable, Achievable, Relevant, Testable.

This is a *really* good idea. We need to go through all of these and
see how they fit the SMART acronym. The one that I'm a bit confused on
is "Specific", since the nature of the advice is not for a specific
application (unlike traditional requirements). Should we drop the
"Disallow illegal characters / Only allow legal characters" because we
can't define what that set of characters is in this document? I
realize that not making it specific makes it difficult to measure and
test, but I also think it's fair to leave share some of that burden
with the framework developers themselves.

> > I just think it could use something else as its source material / organization.

That's a good point - any suggestions here? Again maybe we can connect
off-list about this point

Thanks again Paco!

Cheers,

Rohit

On Tue, Jan 12, 2010 at 3:31 PM, Paco Hope <Paco at cigital.com> wrote:
> On Jan 12, 2010, at 9:23 AM, Rohit Sethi wrote:
> > Many of us have argued that the features of underlying web
> > applications frameworks will make a major impact on the security of
> > the individual applications built on top of them.
>
> This is timely, relative to John Steven's recent discussion (re: ESAPI) that it's not all in the code. There's a ton 
> to be done in the code, but it's not all there.
>
> > I?d like to propose turning this into an OWASP project, but wanted to
> > solicit feedback from the security community prior to turning it into
> > an official project.
>
> I think this is an interesting start, but it seems unfocused. Some of the problems are broad (data confused as 
> commands) and some are highly specific (getting down to the name of the researcher who discovered it). Some seem like 
> they mention technology that's cool without describing the problem that the technology addresses and why that 
> technology is the right answer to the problem. ?Some seem like issues that a framework could offer (pluggable 
> security) and others seem like they are operating system issues (security logs) while still others seem like they are 
> application business logic (login forms and password requirements).
>
> Providing some set of criteria and/or features that indicate compliance with certain best practices is a fine idea. 
> Should you spend your time enumerating flaws and attacks, though, or spend time enumerating correct behavior? I say 
> this because the criteria in this document do both. In some cases they are prescriptive: "use cryptographic random 
> numbers." In other cases, they are untestable negatives: "don't accept illegal characters."
>
> I think each of these criteria should follow a formula. There needs to be some acceptance criteria for what makes the 
> cut to be included in the manifesto, too.
>
> I think each of your criteria should have the same qualities that good software requirements have. They're often 
> remembered using the mnemonic "SMART". Different people assign slightly different names, but I would suggest that, 
> for your purposes, you consider: Specific, Measurable, Achievable, Relevant, Testable. ?I think you can admit 
> criteria to your list that are weak in one of these areas, but if it is weak in two or more areas, it either needs to 
> be expanded or removed (IMO).
>
> You have good and bad examples for each of these, so let me try to demonstrate:
>
> Specific: "illegal characters" is not specific, because what is illegal for one context is mandatory in another 
> context. "Provide an encoding format for every page": that's specific.
>
> Measurable: "Safe file upload" is not measurable (any more than secure web app framework is). Supporting configurable 
> session timeouts is measurable.
>
> Achievable: Safe interaction with unmanaged code is very hard to do. Not to say impossible, but achievability is 
> going to be a key issue. ?Turning on secure cookie flags is obviously achievable.
>
> Relevant: I'm not sure if password policy and "login form" are relevant. If this is a framework we're talking about, 
> application developers tend to manage this in their own business logic, not the underlying framework, right? ?The 
> rest of your list is pretty relevant.
>
> Testable: I'm not sure if the arithmetic overflow protection is testable. ?Other things, like cryptographic 
> randomness is, but even then you have to be careful how you specify the tests.
>
> Lastly, numerous people have created countless lists of flaws, defects, vulnerabilities, attack patterns, security 
> patterns, programming mistakes, design mistakes, and so on. The security world is quite thick with them. I don't see 
> hardly any referenced here, certainly none older than a year or two, despite software security being a 40-year-old 
> field. ?I like your fundamental idea, of having a checklist of capabilities that enables descriptions and comparisons 
> of frameworks. ?I just think it could use something else as its source material / organization.
>
> Paco
> --
> Paco Hope, CISSP - CSSLP
> Technical Manager, Cigital, Inc.
> http://www.cigital.com/
> Software Confidence. Achieved.



-- 
Rohit Sethi
Security Compass
http://www.securitycompass.com
twitter: rksethi