Re: Octave

[Dave Aitel <dave () immunityinc com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
dan () geer org wrote:
> Anyone have an experienced opinion of Octave (cert.org/octave)?
>
> Off-list if you want.
>
> --dan
I went and read it this morning since I'd never heard of it before. My
first impression was that the Air Force or CERT really knows how to
put together a slidepack. It's a professional quality work, and
renders perfectly in OpenOffice.

For everyone else who's never heard of OCTAVE, it's essentially a
security methodology aimed at large organizations (like an Air Force
base, for example). It's not limited to information security issues,
but this is clearly their focus. "Our main goal in developing OCTAVE
is to help organizations improve their ability to manage their
information security risks and to protect themselves." That right
there is a pretty big mission statement, but it doesn't tell you
anything about how they plan to do that. If you guessed "Get everyone
in a circle and be prepared to be fill out some forms together!" then
you are in for a treat. :>


Most telling for me was this slide:

"""
There are many approaches for evaluating information security risk. At
the heart of any approach is an assessment, or evaluation. This slide
defines two common approaches: a tool-based analysis and
workshop-based analysis.
The tool-based analysis normally requires someone to input information
about the organization?s assets, threats, and infrastructure
characteristics into a software-based analysis tool. The tool takes
the information and performs a risk analysis, often based on
proprietary mathematical algorithms. There are usually no restrictions
on who enters the information into the process (often it is a small
group of people) or on how they collect the required information. The
interaction and number of people required by this type of analysis is
small.This approach can be quick (after the initial information is
entered into the tool), but it relies on only a few perspectives. The
organization is also placing trust in proprietary analysis algorithms
that might not be well understood by the organization?s personnel.
A workshop-based analysis requires the participation of many people to
build an understanding of assets, threats, and characteristics of the
infrastructure. A small group of people (an analysis team) leads the
process and gathers information using interviews or workshops. The
analysis team reviews and analyzes the information that has been
gathered and creates mitigation plans. Decision-support tools can be
used to assist the analysis team, but the analysis team is responsible
for making all decisions. This approach involves many staff members in
the organization and can be time intensive. However, the people in the
organization make the decisions and understand why the decisions have
been made.
OCTAVE is a workshop-based approach.
"""

We did a number of these at @stake, and I personally didn't find them
to be of value. Workshops have a number of built in problems:
o People lie to you. Often, people won't know the answers at all, but
will still pretend to to look good. In many cases you will get
conflicting information simply because people don't really know what
they're talking about. You can spend forever tracking down the truth
here. What this means is that at the end of the process you don't have
hard evidence and you don't know how reliable your results are.
o Workshops are hugely expensive for what they produce. You're trying
to get a meeting with the CSO, CISO, CEO, various levels of
management, and the actual technical staff. This involves a huge
amount of effort even for a small organization, and is typically going
to be not worth it. The loss of productivity is mind boggling when you
add it up.
o Workshops draw weak conclusions. I'm not sure why this is, but my
experience with them tells me that overall, we didn't end up telling
people anything they didn't know. A good process will, sometimes at
least, produce results that surprise you. Workshops never will.
Perhaps consensus based brainstorming is not a replacement for
leadership or individual knowledge.

So to sum up: I feel that OCTAVE and things like it are a huge waste
of time. This might not be the answer you were hoping for, but it's my
opinion based on having done things like it and having read the
materials presented on the website.

- -dave
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (MingW32)
 
iD8DBQFEaecUtehAhL0gheoRAicGAJ478coHRs2u454npuOGSZTYzz9SUgCfSdQl
++FXiA+J2Rsu3NwAzyvdHmU=
=9hdW
-----END PGP SIGNATURE-----