UMR SAMOVAR -...

17
VETO’08 (19-20/3/2008) CIRM, Marseille Luminy, Université de la Méditerranée E-voting verfication problems across the world J. P. Gibson, E. Lallet, J-L. Raffy Le département LOgiciels-Réseaux (LOR) UMR SAMOVAR (Services répartis, Architectures, MOdélisation, Validation, Administration des Réseaux) [email protected] LOR-SAMOVAR 2 VETO 08 E-voting: worldwide Nedap in Ireland USA: iVotronic in South Carolina EU: vote counting in Scotland

Transcript of UMR SAMOVAR -...

Page 1: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

VETO’08 (19-20/3/2008)CIRM, Marseille Luminy, Université de la Méditerranée

E-voting verfication problems across the world

J. P. Gibson, E. Lallet, J-L. RaffyLe département LOgiciels-Réseaux (LOR)

UMR SAMOVAR(Services répartis, Architectures, MOdélisation, Validation, Administration des Réseaux)

[email protected]

LOR-SAMOVAR2 VETO 08

E-voting: worldwide

Nedap in Ireland USA: iVotronic in South Carolina

EU: vote counting

in Scotland

Page 2: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR3 VETO 08

E-voting: worldwide

Nedap in Ireland

They said: “The software remains under continuous development and is not of sufficient quality to enable its use to be confidently recommended.

Even if it can be demonstrated to work in most situations, the processes and documentation that underpin the design and development of this software are insufficient to enable its reliability to be assured with the necessary levels of confidence by analysis or inspection of the source code.

Functional testing has revealed programming errors and suggests the possible existence of others, thus further reducing confidence in the software.”

The commission was unable to recommend use of

Election Management Software:

LOR-SAMOVAR4 VETO 08

E-voting: worldwide

USA:

iVotronic in South Carolina

Attempts to vote for one candidate

on the iVotronic were repeatedly

changed to an opposing candidate

on the voter verification screen.

They said: “such vote-flipping is due to calibration errors — touches on the screen are simply registering incorrectly There is a well-defined, simple, 15-step process that poll workers can follow in order to re-calibrate the screen.”

Page 3: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR5 VETO 08

E-voting: worldwide

EU: vote counting

in Scotland

Officials said:

“The system counted the votes but was unable to consolidate the data

constrained in the machine before printing out the result”

140,000 ballot papers rejected

LOR-SAMOVAR6 VETO 08

E-voting: its not a joke

Voting is not a

joke, it is a

privilege and a

responsibility.

It is not to be

taken lightly, it

is not

entertainment,

it should

require some

thought.

Page 4: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR7 VETO 08

Structure of Talk

The CEV (Commision for electronic voting) in Ireland: verification after system delivery

The EU problem: verifying machines against international “recommendations” that are inadequate

The USA problem: enforcing evolving legal standards for voting machines

Generic solution: verification-based software engineering processes using formal methods (where appropriate)

Work in progress: feature-oriented domain analysis for Software Product Line

LOR-SAMOVAR8 VETO 08

E-voting in Ireland

The Irish government invited tenders for the supply of an electronic voting system, which led to the selection of a system made by UK/Dutch company Nedap/Powervote. [June 2000]

TimeLine

Page 5: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR9 VETO 08

E-voting in Ireland

Electronic voting was “successfully trialled” in the general election and Nice referendum in seven constituencies in Ireland. [October 2002]

TimeLine

LOR-SAMOVAR10 VETO 08

E-voting in Ireland

Documentation released under the Freedom of Information (FoI) Act reveals that there were “serious inconsistencies”with the counts in two of the constituencies in which e-voting was piloted [late 2002].

TimeLine

Page 6: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR11 VETO 08

E-voting in Ireland

Widespread Critical Media Coverage of e-voting (in Ireland) [2003]

TimeLine

LOR-SAMOVAR12 VETO 08

E-voting in Ireland

The Independent Commission on Electronic Voting and Counting at Elections (known as the CEV -"Commission on Electronic Voting") was established by the Government of Ireland [March, 2004]

TimeLine

Page 7: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR13 VETO 08

E-voting in Ireland

The Commission of five members was required by its terms of reference to report on the electronic voting and counting system that has been chosen for use at elections and referenda in Ireland. [April 2004 – September 2006]

TimeLine

LOR-SAMOVAR14 VETO 08

E-voting in Ireland

Reporting: Secrecy, Accuracy & Testing of the Chosen E-Voting System

Interim Report - April 2004

The Commission was not able to satisfy itself as to the accuracy and

secrecy of the system for the following main reasons:

Software Versions – final version “not available”

System Testing - insufficient

Source Code - did not obtain access to the full source code

Accuracy – software …impossible for anyone to certify its accuracy

Secrecy - can let voters identify themselves in context of corruption or

intimidation

TimeLine

Page 8: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR15 VETO 08

E-voting in Ireland

Reporting: Secrecy, Accuracy & Testing of the Chosen E-Voting System

First Report - December 2004

Recommends the development of a programme for

software assurance and system testing

TimeLine

LOR-SAMOVAR16 VETO 08

E-voting in Ireland

Reporting: Secrecy, Accuracy & Testing of the Chosen E-Voting System

Second Report - July 2006

Unable to recommend the election management software

used to prepare elections and to aggregate and count the

votes.

A need for comprehensive, independent and rigorous end-

to-end testing, verification and certification … of the entire

system as proposed for use in Ireland.

TimeLine

Page 9: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR17 VETO 08

E-voting in Ireland

Software … has not been developed in accordance

with any recognisable standard process.

Full analysis of the software may not be possible

without a specification of what its behaviour should

be. However, it is not clear whether such a

specification exists…

LEGALLY SPEAKING: The commission emphasises that its conclusion is not based on any finding that the system will not work, but on the finding that it has not been proven at this time to the satisfaction of the commission that it will work.

LOR-SAMOVAR18 VETO 08

E-voting in Ireland

Current Status:

The government has spent €52 million on electronic voting machines and spends approx. €800,000 per annum to store the machines.

Bertie Ahern defended the flawed system and has said in the Dáil, that elections after 2007 should be done without “stupid old pencils”.

Approximately €0.5m is expected to be spent improving the software.

Ahern has defended the system despite public scepticism and opposition from within his own party on the basis that having spent the money, it would cause “loss of national pride” if the system were scrapped.

Page 10: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR19 VETO 08

E-voting: the EU recommendations

The Multidisciplinary Ad Hoc Group of Specialists on legal, operational and technical standards for e-enabled voting was set up by the Council of Europe in early 2003

“. . . to develop aninter-governmentally agreed set of standards for e-enabled voting, that reflect member states differing circumstances, and can be expected to be followed by the ICT industry.”

LOR-SAMOVAR20 VETO 08

E-voting: the EU recommendations

The document they produced acknowledges that it cannot be judged in isolation.

It states that it should respect: “the obligations and commitments as undertaken within existing internationalinstruments and documents, such as [. . .]”

The list of 12 instruments that follows - though it is clearly not meant to be exhaustive - covers a diverse range of documents, including the Code of Good Practice in Electoral Matters

Page 11: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR21 VETO 08

E-voting: the EU recommendations

This inter-related set of complex documents is analagous to a software system which has evolved over time, in response to ever changing sets of requirements.

The system depends on a large number of other systems, and the environment of the system is not clearly understood.

We propose a re-engineering of these standards, but note that this needs participation from a wide range of experts.

However, there is currently no better alternative that could be adopted in place of the European standards: “no requirements catalogue existsthat expresses the requirements for e-voting systems with enough precision to be checkable”. [McGaley]

LOR-SAMOVAR22 VETO 08

E-voting: the EU recommendations

Europe provides numerous examples of countries trying to follow the standards but still failing to produce acceptable solutions.

A main problem is that the standards can be more of a hindrance than a help:

•Over and under specification•Incompleteness•Inconsistency and contradictions …

Page 12: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR23 VETO 08

E-voting: what about the USA?

In the USA, multiple layers of federal, state, and local laws, policies, regulations, and procedures must be followed when running elections.

The recount of votes in Florida during the 2000 presidential election exposed many problems with the traditional voting systems.

LOR-SAMOVAR24 VETO 08

E-voting: what about the USA?

To address the concerns, the Help America Vote Act (HAVA) was signed into law two years after “Florida”.

Currently, most election jurisdictions use systems that are required to conform to the 2002 standards developed by the Federal Election Commission (FEC, 2002).

The standards present a certification procedure involvingtesting by an ITA and most jurisdictions are legally forbidden to use uncertified systems.

Federal government is thus responsible for the testing, certification, decertification, and recertification of voting equipment.

This responsibility has been assigned to the Election Assistance Commission (EAC), an independent commission established in 2003.

Page 13: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR25 VETO 08

E-voting: what about the USA?

The Election Science Institute Project Director, Steven Hertzberg:

“Help America Vote Act has not stimulated sufficient competition among voting-equipment manufacturers…I don't want legislation to stipulate a solution, I want legislation to stipulate a set of requirements based on the needs of stakeholders. And then I want to be able to go out to private vendors and say, 'I need this. Build it.'”

LOR-SAMOVAR26 VETO 08

E-voting: what about the USA?

Sounds similar to the Irish problem, where the CEV reported:

“In the case of the chosen system, many of these requirements were largely predetermined by the fact that an existing design of electronic voting system was adopted and adapted for use in Ireland and that theirexistence was thus already implicit or explicit in that design.”

Page 14: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR27 VETO 08

E-voting: what about the USA?

Ray Martinez, former vice chairman of the Election Assistance Commission which administered $3 billion in federal funding under the 2002 Help America Vote Act, summarised the problem by stating:

"When you add so much complexity - federal mandates, state mandates, new equipment, statewide databases - to an endeavor so dependent on human interaction, you're bound to get mistakes.”

LOR-SAMOVAR28 VETO 08

E-voting: what about the USA?

The American standards call for three levels oftests to be performed on voting systems to ensure that the

end product is fit for purpose:

1. Qualification tests to be performed by ITAs designated by the National Association of State Election Directors;

2. Certification tests to be performed by the State; and

3. Acceptance tests to be performed by the jurisdiction acquiring the system.

Page 15: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR29 VETO 08

E-voting: what about the USA?

So Independent Test Agencies verify the machines before use:

Despite this logical, layered approach to verification, there have been many instances of certified election systems being “broken” (see following slides for some examples)

If systems that meet the standards can be induced to provide inaccurate or unreliable results, is the problem that the standards are poor or is the problem that the verification processes are inadequate?

LOR-SAMOVAR30 VETO 08

E-voting: what about the USA?

North Carolina 2004: Approx. 4,500 votes were lost because officials believed a computer storing ballots electronically could hold more data than it did. Impact: election compromised but results stand

Authorities: There is no way to retrieve the missing data. That is the situation and it's definitely terrible. (But the result would still be the same)

The Media: The point is not whether the votes would have changed things, it's that they didn't get counted at all

Who identified the problem: An election worker noticed that the system’s central controller displayed an error message, “Voter Log Full”, however, the display

continued to increment the number of ballots cast.

Where was the problem: UniLect Corp., the maker of the county's electronic voting system, told them that each storage unit could handle 10,500 votes, but the limit was

actually 3,005 votes

Further Analysis: Non-expert election workers took the incrementing of the number of ballots cast to be evidence that votes were still being recorded.

Page 16: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR31 VETO 08

E-voting: what about the USA?

Florida 2006: In Sarasota County, machines failed to record 18,000 possible votesImpact: election officials – “probably incorrectly” - declared Buchanan the winner over Jennings, by 369 votes in a race with 238,249 votes cast.

Authorities: Although undervoting in absentee ballots was 2.5 percentfor this race, it was about 15 percent for votes cast on

electronic voting machines.

The Media: electronic machines usually register an undervote of <1%

Who identified the problem: Election officials knew that there were issues with the machines, based upon pre-election day voting, and elections officials called the

problem “critical”. Officials were concerned enough to ask poll workers to caution

voters on Election Day to be careful not to “miss the race”.

Where was the problem: the congressional race was “easy to miss” because of its placement at the top of the second screen of choices, above a colored header

introducing the state office races that followed, and that the ballot layout and design

were thus unclear and confusing.

Further Analysis: "banner blindness" is a well documented HCI problem

LOR-SAMOVAR32 VETO 08

E-voting: what about the USA?

Ohio 2004 - Sandusky County: some ballots in nine precincts were counted twice.

Impact: Legal Action between citizens and state resulted from loss of confidence in e-voting system arising out of this and other problems (including a vote count of -25million!).

Authorities: Many election officials from different counties resignedMedia: Claims of conspiracy and fraud

Who identified the problem: election officials

Where was the problem: What appeared to be an over-vote resulted when a computer disk containing votes was accidentally backed up into the voting machines twice by an

election worker.

Further Analysis: It is entirely wrong to put this down to human error and to say it isn’t a software problem. The requirement that a vote gets counted only once should have

been enforced by the software even after the human error.

Page 17: UMR SAMOVAR - alexis.bonnecaze.perso.luminy.univ-amu.fralexis.bonnecaze.perso.luminy.univ-amu.fr/VETO/Veto08-IrelandUKU… · Generic solution: verification-based software engineering

LOR-SAMOVAR33 VETO 08

E-voting: How it should be done

Professional Software Engineering on all e-government projects

Formal methods for e-voting, where appropriate:

•Interface design•Vote storage•Feature interactions in requirements models

Software Product Line (feature domain analysis)