Belated comments on David Rice

[jackadaniel at gmail.com (Jack Daniel)]

Better late than never-

David Rice said some things in episode 160 that just don't work for
me, and I feel compelled to address a few of them.  Mr. Rice is a big
proponent of some kind of testing/certification framework for software
to insure security.  This sounds good, but the devil is in the
details, especially his.

He suggests the testing will need to start out with a low set of
standards, and improve/evolve over time, getting tougher until they
are a real measurement and enforcement of security.  Again, this
sounds good, but it is where things start to unravel.  He points to
IIHS and NHTSA testing as models; according to him they started nearly
useless and evolved over decades- which is true to an extent, but
fundamentally unsafe cars still pass the tests, and there are many
real dangers which are not considered in the tests and ratings.  Only
a specific (and well known to the manufacturers) set of tests are
performed, this leads to building cars to pass the test, not
necessarily actually be safe (hey, that sounds familiar, doesn't it?).
 Let's blow NHTSA/IIHS testing out of the water, shall we?

I love Jeeps, and have had several over the years.  From the first CJs
through the current Wranglers, and it's easy to make the case that
they are all horribly "unsafe".  The old ones were underpowered, but
short, skinny, and prone to rollovers. As they have grown longer and
wider, the improved stability has been offset by increased power so
they are still rollover-prone.  Driven properly, the risks can be
minimized, but that means trusting the end-user (we're screwed).  But,
they "pass" the tests.  Maybe not the highest safety ratings, but
passing.  If the remnants of what we call Chrysler didn't know both
the questions and the correct answers before they sent their Jeeps in
to take the tests, I wonder if they would pass.

You need look no further than the lists of safety recalls
(http://www-odi.nhtsa.dot.gov/recalls/recallmonthlyreports.cfm) to see
how often and how badly this testing fails, and the myriad of things
not tested which endanger you, me and our families (real world injured
or dead on the road, no buffer overflow nuisances).  Improper child
seat mounting, mis-welded steering columns, overheating electronics,
motorcycles helmets which fail to meet the standards, tow bars (for
above mentioned Jeep Wranglers), leaking fuel filters- and that's just
the first page and a half (of eighteen pages) in the report for the
single month of July 2009.

Think about it, how can slamming a car into a barrier tell you that
the lower control arms are formed so that they will trap sand, salt,
and moisture- then rust out in two years and cause your suspension to
separate when you hit a pothole "just right".  That takes a level of
inspection and review we haven't gotten to yet, and probably never
will.  And I can shine a light on the arm and hit it with a hammer-
try that with software testing.

One of my favorite examples of not testing for real-world safety is
the moose.  In limited parts of the world moose collisions are
extremely common, and they are extremely dangerous wherever they
happen.  One of the primary problems is that cars are simply not
designed to withstand an impact from the front against the a-pillars
(the uprights between the front doors and the windshield), and they
collapse easily.  It is a known "defect", but it is not tested for,
and the cost of correcting the defect is not justified- and people
die, avoidably, every year because of it.

Has that dampened your enthusiasm for automotive testing as a model
for software assurance?

He also mentioned public pressure and demand for safer vehicles as
supporting the improved safety, but the two highest profile news
stories I recall about auto safety were both fraudulent hype.  First
we had the "runaway Audis", which supposedly accelerated out of
control on their own.  Audi maintained that the drivers were simply
mistakenly stomping on the gas instead of the brake in all of the
events, but 60 Minutes did a segment where they showed the engine of
an Audi racing away "by itself" and drove public panic and outrage
(costing Audi and their dealers untold millions).  The "runaway" car
60 Minutes showed?  It had its transmission rigged to drive the
throttle linkage and artificially accelerate the engine; Audi fixed
the operator error by retrofitting the brake/shift interlock system we
all take for granted on cars now.  The other big one involved GM
trucks with fuel tanks outside of the frame rails, something that WAS
unsafe, but Dateline NBC felt the need for a bit of drama to make the
point.  They staged side-impact collisions with the trucks, but didn't
get much more than a little fuel leakage.  Needing more drama to sell
their schtick, they attached small rocket motors to the fuel tanks and
rigged the test to guarantee spectacular fires, in the aftermath of
the stunt, the real dangers of the design were overshadowed by the
fraudulent reporting.  I have no need for Fox News, CNN, MSNBC, or any
other pack of screaming dimwits driving the discussion about software
security.  If you like the idea, ask them what a "hacker" is and get
back to me.

Moving on...
When he said "cars aren't as complicated as software", I lost it.
Clearly, he misspoke; surely he cannot believe a automobile where an
entire Windows Media Center entertainment system is a fscking
afterthought tossed into the dashboard is somehow LESS complicated
than the afterthought itself.  Cars have had microprocessors for
decades, and hydraulic computers for decades longer than that.  (If
you think transistors are clever, try doing similar switching of
forces in a high-temperature hydraulic environment which makes your
car go down the road at varying speeds, that's what automatic
transmissions have done for over half a century).

Finally (for this rantbuttal), how the !@#$ is the idea of a
testing/certification framework which starts out merely enforcing
current expectations and slowly evolves and becomes more strict over
decades (with inevitable stumbles and false starts) until it is as
[fundamentally flawed] as automotive crash testing a unicorn-inducing
wonderful idea when he suggests it for software...

but the exact same thing is a horrible and destructive idea if it is
applied to cardholder data and we call it PCI?  Hold a mirror up to
all of his arguments before you answer that.  He did a phenomenal job
of rebutting himself when he launched into his PCI rant- so I'll stop
there.

Having beaten this topic worse than a crash-test dummy on Mythbusters,
I now return you to your regularly scheduled Saturday afternoon.


Jack




-- 
______________________________________
Jack Daniel, Reluctant CISSP
http://twitter.com/jack_daniel
http://www.linkedin.com/in/jackadaniel
http://blog.uncommonsensesecurity.com