Re: Diebold Admits Audit Logs in ALL Versions of Their Software Fail to Record Ballot Deletions

[David Farber <dave () farber net>]

Begin forwarded message:

From: Joseph Lorenzo Hall <joehall () gmail com>
Date: March 20, 2009 6:38:58 AM EDT
To: dave () farber net
Cc: Abe Singer <abe () oyvay nu>, "Michael O'Dell" <mo () ccr org>
Subject: Re: [IP] Re: Diebold Admits Audit Logs in ALL Versions of  
Their  Software Fail to Record Ballot Deletions

Hi Dave, responding to O'Dell and Singer's comments:

This is a vast, fascinating subject.

> How much more is it going to take for there to be a requirement
> for all voting system software be published for public scrutiny?

Well, as you can imagine it's not as simple as that.  It's not as if
mandating disclosure will change the quality of this software
overnight; there are a slew of other things that need to be in place
for that kind of a move to be effective.  Not to mention that it's not
just a matter of software -- although that's a big piece.  With
multi-level circuit boards and FPGAs on these things, there's really
no practical way to "verify" them... many are working on that.  (Plus,
if you find a problem in voting code, it takes about a year --
optimistically -- to get through the federal and state certification
process and make it in fielded systems... it's just not responsive to
changing things quickly.)

Also, a point I raised in a paper from 2006 [1] -- which is remarkably
aging well -- is that we need also be a tad concerned about unilateral
moves to disclose source code or we might run afoul of eminent domain
for intellectual property (trade secrets, here).  To boot, software
*written to be disclosed* is written very differently than software
that isn't... so you can imagine with hundreds of thousands of lines
of code, there's quite a bit of due diligence to be performed.

In that 2006 piece, I argued that limited disclosure to experts could
go a long way towards the promise of source code disclosure and that
we should put manufacturers on notice that past a specified date in
the near future, they will have to disclose elements of their systems
more widely.  I think this has been validated by the very controlled
release of systems and source code to a number of us in California
(the Top-To-Bottom Review), Ohio (the EVEREST review), Florida (the
many SAIT reviews), New Jersey (to Andrew Appel under court order) and
a number of other cases.

> or we could just use paper.

Humboldt did use paper.  And, in fact, the move has been in many
jurisdictions to go back to Scantron-like precinct-based optical
scanner technologies.  These are great because the voter marks a paper
record that is then scanned and interpreted and retained for 22 months
according to federal law.

Maybe the call here is to go all paper?  Well, that's increasingly
impractical for a variety of reasons.  First, we have very complicated
ballots in the US with state, local and federal races on the same
ballot.  When combined with the mess that is our primary system, many
*individual precincts/polling places* can have dozens of different
ballot styles available.  Counting these in a timely fashion can be
very tough.

The real answer here is robust post-election auditing, which hasn't
been as widely adopted as independent paper records.  The future lies
in what we call "risk-limiting" audits where we retally, by hand, a
random sample of ballots/precincts/etc.  Humboldt employed a new model
of post-election auditing where a separate software system was used to
retabulate all ballots... and they hit gold and made our elections
just a bit more safe.

> I wonder, how did this product receive certification in the first
> place when this particular flaw violated federal certification  
> standards?

I'm not certain where to begin.  We've written extensively on the
shortcomings of the standards and certification process [2,3].

First, certification of systems has a long history that's mostly
disappointing (Peter Neumann, a PI in our ACCURATE NSF e-voting
center, reminds me of just how well Common Criteria and Orange/Red
book certification works/worked).

It's definitely true that all the machines in use today were certified
to outdated standards (in the Humboldt case, to ***1990*** standards).
And we know pretty well that the testing laboratories that are tasked
with the certification process haven't done a very good job in the
past.  This case is interesting because it's not instantly clear that
these are all violations of either the 1990 or 2002 standards, mostly
because those standards aren't written with test cases and other
bright line evaluative measures.  And, of course, more than one
certifier missed this... the CA SoS missed it too.

The biggest and most disturbing realization for many of us has been
recognizing that many voting system manufacturers are using the
certification process as a QA process, either in lieu of their own or
as a more robust version ("Oh, they'll test that in certification so
we don't have to.").  This, of course, compounds things as the testers
have a moving target that changes during certification and they truly
are faced with systems that often don't work very well at all.

It's a dark time, but there are hopes for better models, systems,
business opportunities and safer systems.

best, Joe

----

[1] Joseph Lorenzo Hall. (2006). Transparency and Access to Source
Code in Electronic Voting. USENIX/ACCURATE Electronic Voting
Technology Workshop 2006. Retrieved from
https://www.usenix.org/events/evt06/tech/full_papers/hall/hall.pdf.

[2] Erica Brand, Cecilia Walsh, Joseph Lorenzo Hall, & Deirdre K.
Mulligan. (2005). Public Comment on the 2005 Voluntary Voting System
Guidelines. A Center for Correct, Usable, Reliable, Auditable and
Transparent Elections. Retrieved January 18, 2008, from
http://accurate-voting.org/accurate/docs/2005_vvsg_comment.pdf.

[3] Aaron Burstein, & Joseph Lorenzo Hall. (2008). Public Comment on
the Voluntary Voting System Guidelines, Version II (First Round).
Submitted to the Election Assistance Commission on behalf of ACCURATE
by the Samuelson Law, Technology and Public Policy Clinic. Retrieved
May 20, 2008, from
http://josephhall.org/papers/accurate_vvsg2_comment_final.pdf.

--
Joseph Lorenzo Hall
ACCURATE Postdoctoral Research Associate
UC Berkeley School of Information
Princeton Center for Information Technology Policy
http://josephhall.org/




-------------------------------------------
Archives: https://www.listbox.com/member/archive/247/=now
RSS Feed: https://www.listbox.com/member/archive/rss/247/
Powered by Listbox: http://www.listbox.com