Automating Voting Terminal Event Log Analysis · 2019-02-25 · Automating Voting Terminal Event Log Analysis Tigran Antonyan Seda Davtyan Sotirios Kentros Aggelos Kiayias Laurent

Automating Voting Terminal Event Log Analysis

Tigran Antonyan Seda Davtyan Sotirios Kentros Aggelos KiayiasLaurent Michel Nicolas Nicolaou Alexander Russell Alexander Shvartsman

{tigran,seda,skentros,nicolas}@engr.uconn.edu{aggelos,ldm,acr,aas}@cse.uconn.edu

Voting Technology Research Centerand Computer Science and Engineering Department

University of Connecticut

AbstractIn the interest of auditing election procedures, certainelectronic voting technologies provide monitoring capa-bilities that record select actions undertaken by electionofficials before, during, and after an election process, aswell as the conditions present in an electronic voting ter-minal as the result of its interactions with its environ-ment. In this paper we report on an automated auditingprocess for detecting procedural irregularities for elec-tions employing the AccuVote Optical Scan (AV-OS) ter-minal (manufactured by Premier Election Systems).

Our auditing process is derived from an abstract finitestate model of the AV-OS; this determines, in particular,a correspondence between state transitions and loggedevents that separates expected and “irregular” histories.Automating the detection of these irregular histories haspermitted us to provide detailed election procedure au-dits for full-scale Connecticut elections. We concludethe article with a discussion of the result of the event loganalysis performed within the post-election audit of theNovember 2008 elections in Connecticut.

Additionally, we identify a defect and some deficien-cies in the AV-OS event logging subsystem that can in-terfere with the event log transcript making it vulnerableto manipulation and discuss the effects of these deficien-cies.

This research is funded by the Office of the Secretaryof the State of Connecticut.

1 Introduction

Post-election audits are essential to ensure voter confi-dence in the outcome of elections. Many states mandateaudits and official recounts when the number of votescast for leading candidates are within a close proximity(e.g., within 0.5% in Connecticut), or when the electionresults are disputed. In addition, it is now consideredimportant to conduct random audits regardless of the

perceived outcome [12], and some states mandate post-election audits for all general elections. The broad intro-duction of new, and in some cases untried, electronic vot-ing equipment prompted by the Help America Vote Act(HAVA) [5] underscores the need for independent exam-inations of the election outcomes. Recent research pro-vides strong evidence that the security and dependabil-ity concerns associated with electronic voting technologyare very real (e.g., [11, 10, 13, 15, 6, 7, 14, 4, 8, 9]). Con-sequently, nationwide discussions involving various pub-lic officials and diverse civic groups have been focusingon the need to enable and to perform meaningful elec-tion audits, and in particular, audits of electronic votingtechnologies used in the elections. It is noteworthy thatwhile concerns about the electronic voting terminals per-sist, electronic voting also enables the capture of valuabledata pertaining to the conduct of an election in additionto reporting the final tally. For example, electronic vot-ing terminals are normally equipped with the capabilityto record an event log that contains timestamped entriescorresponding to significant actions performed by the ter-minals and their users. The ability to analyze event logsoffers a valuable capability within any approach to elec-tion audits.

This paper presents our approach to and work on au-tomating the analysis of the event logs of electronic vot-ing terminals. The University of Connecticut VoTeRCenter (Voting Technology Research Center) is an in-dependent entity, and we perform our work within anestablished relationship with the Connecticut SOTS Of-fice, that chartered us to examine and evaluate electronicvoting technology, to assist the State in defining safe useprocedures for the electronic voting terminals, and to per-form audits of technology before and after each election.Our previous work related to audits of technology is pre-sented in [2].

We have implemented a tool that allows for the eventlogs collected in the wake of an election to be methodi-cally analyzed, with the goal of detecting any deviations

1

from the prescribed and recommended use of the votingterminals in the election. On the request of Office of theSecretary of the State (SOTS) of Connecticut, we usedthis tool after the November 2008 election to perform adetailed analysis of the voting terminal audit logs repre-senting over 30% of all precincts in the State. We presenta summary of our findings in this report, with the conclu-sion that detailed event log analysis should be includedas an essential part of any audit process.

We believe the approach to event log analysis pre-sented in this paper is very general. Naturally many elec-tronic voting technologies incorporate monitoring capa-bilities that enable retrieval and examination of the ac-tions performed on and by the system during an elec-tion. These actions are normally recorded in preallo-cated memory space that we refer to as the system’s eventlog. We present our approach in the context of a spe-cific voting terminal, Premier’s Accu-Vote Optical Scan(AV-OS) terminal (firmware version 1.96.9), that is usedin Connecticut elections since 2006. Specifically, wepresent a part of the post election audit process that dealswith the analysis and examination of the event logs col-lected from the AV-OS machines.

The event log of an AV-OS voting terminal is storedin a removable memory card [2]. While the voting ter-minal hardware is identical in all Connecticut precinctsdeploying AV-OS systems, the contents of the memorycards differ depending on the specific elections in indi-vidual precincts. The event log within an AV-OS termi-nal contains a history of actions since the memory cardwas formatted and programmed for the specific precinct.

We note that, for the purposes of this work, we treatedthe AV-OS terminal as a “black box”. The results derivedand presented here are obtained through extensive testingprocedures using the functions provided by the machine,and through observations of how the terminals’ use andbehavior is recorded in its event logs.

This paper emphasizes the importance of including theevent log analysis as a part of a post election audit oftechnology. A careful examination of events and corre-sponding timestamps reveals information about the pro-cedures followed during an election process, includingthe information suggesting improper conduct of an elec-tion or malfunction of terminals.

Related Work: The AV-OS voting terminal has drawnthe attention of a number of research papers that demon-strated security vulnerabilities in the system [6, 8, 9, 3,2]. The report of Hursti’s [6] revealed that the AV-OSmemory card lacks cryptographic integrity checks, thusit can be maliciously manipulated and exploited with thehelp of specialized hardware. These findings led severaljurisdictions employing the system to use tamper-evidentseals in securing the memory card in the terminal. The

Connecticut SOTS commissioned a follow-up study toconfirm Hursti’s findings and identify other vulnerabil-ities. Our study [8, 9] demonstrated that an additionalattack can be successfully launched against the AV-OSeven if the memory card is sealed in. Here the attack ex-ploits a communication port on the machine and by com-mandeering its card writing capability, it transparentlymodifies the contents of the memory card. The same at-tack also exploits vulnerabilities of the machine’s pro-prietary language, called AccuBasic, used for reportingelection results.

Although previous attacks assumed that the firmwareof the AV-OS terminal is a trusted component of the sys-tem, subsequent reports [2, 3] proved this assumptionto be incorrect. In particular the two reports show thatsomeone with physical access to an AV-OS terminal maymanipulate the firmware code in a way that will be un-detectable during election day testing. While [3] demon-strates both benign and malicious manipulations of thefirmware, [2] underscores and presents procedures forpre-election and post-election testing to minimize and/oreliminate any vulnerabilities that potentially may alterthe election results. Our approach leads the way towardsensuring integrity of the electoral process, over unreli-able and untrusted hardware. The results of this workand the techniques developed by us were used in the au-dits of technology in all elections in Connecticut startingin 2006. In this paper we present a new step that en-hances election audit procedures presented in [2] with adetailed examination and analysis of the AV-OS eventlogs.

Contributions: Our goal is to introduce event log anal-ysis procedures that facilitate the efficient and accurateanalysis of the event logs of the AV-OS terminals or anyelectronic voting terminal that provides the event loggingcapability. Our development involves the design and im-plementation of a software tool that automates the eventlog analysis. For this purpose we define and describea rigorous computational model, embraced by any elec-toral process that deploys AV-OS voting terminals. Wemap our model to a state diagram and from here we in-troduce action chain rules that must be satisfied by anyconsistent election process. Based on our extensive test-ing and the analysis of event logs we also discover andreport deficiencies in the actual logging process of theAV-OS terminals. In more detail our contributions are asfollows.

• We develop a model that describes the action chainsand specifies the proper stages in an election pro-cess. Based on this election model we develop astate diagram that represents the flow of actions inthe election process and that identifies acceptable

2

and unacceptable action sequences that should bethe target of the event log analysis procedure.

• We present the types of actions that the event logis able to record, then we show the mapping of theactions to the state diagram and we introduce eventrules that should be followed in an election process.

• The specification of event rules facilitated the devel-opment of an event log analysis tool that reports anyinconsistencies or deviations from the prescribedrules based on the event log data. The tool auto-mates the analysis of a large number of event logs,thus expediting this part of the election audit.

• Based on our experimentation with the AV-OS vot-ing terminal, we report on defects and deficienciesin the action recording procedure used by the AV-OS. In particular we discover a log overflow vulner-ability as well as inadequacies in the date and actionrecording procedures. The defect in the event logoverflow handling may affect the event ordering onthe log printout and may lead to malicious event logmanipulations. The inadequate date recording mayhide the actual execution time of the events. Lastly,inadequate action logging may result in the absenceof reporting of certain events.

• Finally, we present the results and our observationsbased on the application of our automated event loganalysis tool in the actual post-election audit con-ducted after the November 2008 elections in Con-necticut.

We believe that audits of election technology, and inparticular audits of truthful event logs of electronic vot-ing terminals, are crucial in providing timely informationand maintaining the integrity of the electoral process.

We note that our evaluation and tool development wasperformed without any access to internal vendor docu-mentation, source code, or assistance of any kind.

2 The Election Process in Connecticut

Figure 1 illustrates the election process flow when usingan AV-OS terminal. This section describes the electionflow in more detail. We believe this model to be fairlygeneral, however it incorporates an election audit modelthat may be specific to the State of Connecticut (CT).

Before Election Day: Election preparations in CT be-gin at least 30 days prior to the election day. Thememory cards of AV-OS terminals are programmed foreach precinct. The voting terminals also undergo rou-tine maintenance and testing to help prevent failures dur-ing the election. The memory cards are programmed

by a service company contracted by the State. Fourprogrammed cards then are securely transported to eachpolling location. When the cards arrive, election offi-cials conduct pre-election tests on all the voting terminalswith all the cards. Then the officials randomly select twocards. These cards will be used in the primary and theback-up terminals. The two selected cards are sealed intheir respective voting terminals and are set for election.

On Election Day: The summary of the activities thattake place on the election day is as follows.

• Before The Polls Open: On the morning of the elec-tion day, before the polls open, the poll workersand/or registrars of voters need to verify the seals oneach AV-OS terminal, and verify that the machinesare properly initialized. This includes making sureall candidate counters are set to zero, by printing aZero Totals Report.

• While The Polls Are Open: Each eligible voter isentitled to a single ballot that s/he receives once theyare verified against the voter registration database.Once the voter fills the ballot s/he proceeds to feedthe ballot to the optical scanner of AV-OS.

• After The Polls Close: The election officials printthe totals report directly from the AV-OS terminal(the event log can also be printed at this point). Theprinted tape is then delivered to the central tabula-tion process where the totals are computed and re-ported to the Secretary of the State Office for cer-tification. Note that in CT, the central tabulationtool available from the vendor of AV-OS (Premier’sGEMS) is not used. In jurisdictions that use cen-tral tabulation, the results either are uploaded fromAV-OS terminals to a central server where the tab-ulation is performed, or the memory cards with theresults are transfered to the tabulation center, wherean AV-OS terminal is used to upload the results tothe tabulation server. Depending on the state, thecentral tabulation is usually done at the municipalor county level.

Audits: Three independent audits are performed inconjunction with each election (in Connecticut).

• The Pre-Election Technical Audit involves the ex-amination of one memory card, chosen randomlyfrom the four cards supplied to each precinct. Theaudit covers a large subset of precincts. Over thelast three years pre-election audits involved any-where from 25% to 100% of the precincts.

• The Hand Count Audit is a post-election audit thatconsists of complete manual counting of a subset of

3

SECRETARYOF STATE

STATE VOTER

Election ManagementSystem Software 

Audit Report

REGISTRATIONDATABASE

Precinct

GEMS

Audit

Pre‐Election Testing

CHECK‐IN

PrecinctVoter List

RandomSelection

Optical Scanner (AV‐OS)

Pre‐Election Audit

Ballot Box

Voter

Needs aPhoto ID

Ballot RandomSelection

Ballot

(AV OS)

Hand Count AuditBallot Reader(Scanner)

Ballots

Transaction Log

RandomSelection

Memory Card

Tape(Printer)

Post‐Election AuditBallot(s)

Counters

Central Tabulation

Tape

Tally Report

PROVISIONAL &ABSENTEE BALLOTS

Figure 1: Election Process in Connecticut using AV-OS

races in 10% of precincts randomly selected aftereach election.

• The Post-Election Technical Audit involves the ex-amination of the memory cards used in the election,typically covering up to 30% of the precincts.

The memory card audits include examining the cards forproper programming and absence of extraneous or unex-pected executable code [2]. Our current work deals witha detailed analysis of the event logs collected from thevoting terminals used in the election.

3 Event Log Analysis

This section presents our modeling of the voting terminalbehavior that includes the actions recorded in the eventlog and our approach to automating the analysis of thelogs. We implemented a rule-based tool that inputs anevent log generated by a voting terminal and classifies itas either normal, i.e., containing a sequence of events ex-pected within a normal course of an election, or irregular,i.e., containing unexpected events, unusual sequences ofevents, and unanticipated timing of events. The irreg-ular logs can then be further analyzed manually. Ourautomated event log analysis tool was used to perform

a broad audit of the event logs collected from over 30%of the precincts following the November 2008 electionin Connecticut. We present the highlights of the audit inSection 5.

Here we start by presenting the action types recordedby the AV-OS voting terminal in the event log, the statesthat a terminal may go through during the election pro-cess, and the actual traces produced in the event log. Theentire system is modeled as a finite state machine that de-termines the language of valid log sequences. The analy-sis tool augments the basic finite state machine with timepredicates on the transitions to capture the time sensitiv-ity aspect and to model when transitions are enabled ordisabled. This is the basis on which the log analysis toolis built. The section closes with a brief description of theactual tool.

3.1 Actions Recorded in the Event LogTable 1 presents the action types recorded by AV-OS inthe event log along with a brief description. The eventlog has action entries and date entries. Most action en-tries are solely characterized by a name and a time ofoccurrence (no date). Some action entries, i.e., INITIAL-IZED and SESSION START carry an additional date storedin the adjacent date entry.

4

Action Name Action DescriptionAUDIT REPORT Appears when an Audit Report is printed.BAL COUNT END After the ender card is inserted in an election, this action appears.BAL COUNT START Appears when the first ballot is cast in an election.BAL TEST START Records the beginning of a test election.CLEAR COUNTERS Appears when the counters are set to zero.COUNT RESTARTED Appears if the machine is reset during an election, after at least one ballot is cast.DOWNLOAD END Recorded during the programing of the card using GEMS, when the download of data is ended.DOWNLOAD START Recorded during the programing of the card using GEMS, when the download of data is started.DUPLICATE CARD Appears when a card duplication takes place. Present in both the master card and the copy.ENDER CARD Records when an ender card is inserted, signifying the end of an election.INITIALIZED The 1st action that appears in the Event Log. Date action that appears when one programs the card.MEM CARD RESET A memory card reset returns a card in ’not set’ status, if it was set for election.OVERRIDE Records an override by a poll worker. Used for the insertion of overvoted ballots in CT.POWER FAIL If the machine is unplugged or a power failure occurs, this action is recorded.PREP FOR ELECT Recorded when the card is set for election.SESSION START Date action. Appears every time you reset the machine.TOTALS REPORT Appears when a Totals Report is printed.UNVOTED BAL TST Appears when an unvoted ballot test is performed.UPLOAD END When an upload is completed, this action is recorded.UPLOAD ERROR Appears when an upload error is detected.UPLOAD STARTED Marks the beginning of an upload.VOTED BAL TEST Appears when an voted ballot test is performed.ZERO TOT REPORT Appears when a Zero Totals Report is printed.

Table 1: Event log action types

3.2 AV-OS State Diagram

For reasons of security, memory cards are sealed in thevoting terminal prior to pre-election testing and for theduration of the election. Thus we model the voting ter-minal with the inserted cards as a single system from thetime the card is sealed in. Figure 2 illustrates the statesthe system goes through, from the time a card is pro-grammed, until the election is closed and the results areprinted.Remark. The states Blank Card, Election Loaded, SetFor election, Print Totals Report, and Election Closedare the states the machine returns to when it is restarted.These states correspond to the first five states of Fig-ure 1 in [14]. Our diagram does not include the last twostates presented in the same work, since in Connecticutthe results are not uploaded and thus the machines shouldnever enter those states.

Next we provide a detailed description of the transitionfunction. For each state, we indicate what changes takeplace as a result of a user action and which events areplaced in the event log.

3.2.1 States, Actions and Event Log

Each paragraph of this section describes a state and thetransitions that can be followed as a result of user ac-

tions. Each paragraph is associated with a figure. In thefigure, states are represented with ovals and actions withboxes. Arrow heads indicate the flow from the originat-ing state, through the action box and to the resulting state.Each action box is annotated with a tuple 〈a‖b‖c〉 wherea is the user action, b represents the ensuing sequence ofmachine actions, and c captures the sequence of loggedevents.

Blank State. The transitions of the blank state areshown in Figure 3. This state represents a memory cardthat is formatted but contains no election information.One needs a blank card in order to program it usingGEMS, after which the card will contain valid electiondata. This programming also inserts a sequence of eventsin the (initially empty) event log: INITIALIZED, DOWN-LOAD START, CLEAR COUNTERS, DOWNLOAD END.

Loaded Election State. Figure 4 describes the Elec-tion Loaded state. Cards should be in this state whenthey first arrive at the precincts. At this state test elec-tions can be performed or the card can be cleared in or-der to be reprogrammed. According to Connecticut elec-tion procedures pre-election testing is performed on allcards at the precincts. The entity programming the cardswill also perform pre-election testing before it ships the

5

Figure 2: AV-OS State Diagram

cards. From this state, a user can perform the followingactions: unvoted ballot test, voted ballot test, test elec-tion, clear the memory card, print test results or reset themachine. Most of these actions result in the insertion ofa corresponding event in the event log. The two notableexceptions are the voted and unvoted ballot tests.

Voted Ballot Test. Figure 5 illustrates the actions thatcan be initiated during the voted ballot test. The voted

ballot test only accepts a ballot with all its bubbles filledin. If the ballot has some empty bubbles, the voting ter-minal will report which bubbles were not filled. In all thetest states, if one resets the terminal, the card returns tothe loaded election state. For the VOTED BALLOT TEST,the same result can be achieved by stopping the test orpressing the no button. Note that the first time a ballot istested, the voted ballot test event is recorded in the eventlog.

6

Figure 3: Blank State

Figure 4: Loaded Election State

Unvoted Ballot Test. Figure 6 illustrates the actionsthat can be initiated during the unvoted ballot test. Theunvoted ballot test only accepts a ballot with all its bub-bles empty. If the ballot has some non-empty bubbles,the terminal will report which bubbles were filled. Onceagain, if the machine is reset, the card returns to theloaded election state. The same will happen if the test isstopped via the no button. The first time a ballot is tested,the unvoted ballot test event is recorded in the event log.

Test Election with Zero Counters. Figure 7 describesthe zero counters state and its transitions. We distinguishthe test election with zero counters and the test electionwith non-zero counters, as there is a small differencein the actions that can be performed between them, andsome actions lead to the insertion of different sequencesin the event log. As before, resetting the voting terminalreturns the card to the loaded election state. The samehappens if one inserts the ender card. Note that the eventslogged in the two cases are completely different. Cast-ing a ballot in this state will cause a transition to the testelection with non-zero counters state. It also results inthe insertion of a ballot test start event in the event log.Finally in this state one can print a zero totals report.

Test Election with Non-Zero Counters. Figure 8 de-scribes the test election with non-zero counters state.Again, resetting the voting terminal returns the card inthe loaded election state. The same happens if one in-serts the ender card. Note that the events logged in thefirst and second case are different.

Set for Election with Zero Counters. Figure 9 de-scribes the ‘set for election with zero counters’ state.This state can be reached from the ‘election loaded’ stateby using the set for election action. A card should be inthat state the morning of the elections, before any bal-lots are cast. A reset of the terminal at this point will notreturn it to the ‘election loaded’ state. Instead, the termi-nal will remain in the current state and print a zero totalsreport again. Casting any ballot, either with an overrideor normally, moves the card to the ‘set for election withnon-zero counters’ state and adds the BALLOT COUNTSTART event in the event log. An ender card ends theelection, causing the insertion of the following sequenceof events: ENDER CARD, BALLOT COUNT START, BAL-LOT COUNT END. At this point the voting terminal willstart printing the totals report and the card will enter the‘print totals report’ state.

7

Figure 5: Voted Ballot Test

Figure 6: Voted Ballot Test

Set for Election with Non-Zero Counters. Figure 10details the ‘set for election with non-zero counters’ state.The card remains in this state as long as the election isongoing. If the terminal is restarted, the card remains inthe same state and this sequence of two events is insertedin the log: SESSION START, COUNT RESTARTED. Vot-ing terminals should not be restarted during the electionsaccording to Connecticut election procedures. If electionofficials override the machine to cast a ballot, an OVER-RIDE event is recorded. Since the overrides depend onlyon the voters, in a large precinct the event log could con-tain many OVERRIDE events. An ender card will end theelection, causing the insertion of the following sequenceof events: ENDER CARD, BALLOT COUNT END. At thispoint the machine prints the totals report and the cardenters the ‘print totals report’ state.

Print Totals Report. Figure 11 describes the transi-tions of the ‘print totals report’ state. If one restarts theterminal, the card will remain in the same state, and itwill keep on asking whether the operator wants to printa totals report or not. If one says yes, a report is printedand the terminal returns to the same dialog. If one de-clines the dialog and only in such case, the terminal en-ters the ‘election closed’ state. If at least one report wasprinted since the terminal was last restarted, a TOTALSREPORT event will be logged in the event log. Note thatthis makes it possible to have a report printed without anevent TOTALS REPORT entered in the event log. This is adefect in the design of AV-OS and is further analyzed insection 4.2.

Election Closed. Figure 12 describes the ‘electionclosed’ state. From the ‘election closed’ state one canresume the last election that ran on the card. Such anaction results in the following sequence of events beingrecorded: PREP FOR ELECTION, COUNT RESTARTEDand BALLOT COUNT START. The latter appears on thefirst ballot cast on the new election. While on the ‘elec-tion closed’ state, one can print more totals reports. Alsoone could perform a memory card reset action and re-turn the card to the ‘election loaded’ state. Finally onecan clear the card in order to produce a blank card forreprogramming.

3.3 Time Sensitive RulesThe election procedure described in Section 2 is timesensitive. Hence for the purpose of event log auditing, itis important to create time sensitive rules. The event logcan be used as a forensic trace with the necessary time in-formation in order to determine which actions were per-formed at each step of the election process. We highlightbelow the main time intervals characterizing the electionprocess in Connecticut.

Card Programming and Pre-Election Testing. InConnecticut, the programming of the cards is taken careof by an external entity based on the election data pro-vided by the State. The programming usually starts threeweeks before the elections and is completed in a weekfor almost all precincts. Moreover it is expected that pre-election testing is performed on all the cards prior to their

8

Figure 7: Test Election with Zero Counters

Figure 8: Test Election with Non-Zero Counters

shipment to the precincts. All the events related to theinitialization of a card along with some pre-election test-ing events should occur within that time frame.

Pre-Election Testing and Setting for Election in thePrecincts. Cards arrive in the precincts any time oneweek to two weeks before the elections. The electionofficials perform pre-election testing on all the cards thatarrive. They pick two cards (randomly), seal them in thetwo AV-OS machines that are available in each precinct,and set them for election. At this time interval we expecta number of events related with pre-election testing anda single PREP FOR ELECTION event. After being set forelection, one would expect the cards to not contain anyevents until the day of the election.

Election Day: The election day imposes strict timelimitations on when and what actions can be performed.According to the election process we expect to seethe following: from 5:00 to 6:00 AM election officialsshould switch on the AV-OS terminal to be used in theelection and produce a zero totals report. This meansthat we expect to see a session start (SESSION START)event about an hour before the polls open, followed bya zero totals report (ZERO TOT REPORT) event. We ex-pect to see the ballot count start (BAL COUNT START)

event after the time the polls open. This can be followedby a sequence of override (OVERRIDE) events. Finally,by the time the polls close, we expect to see the endercard (ENDER CARD) event, followed by a ballot countend (BAL COUNT END) and a totals report (TOTALS RE-PORT) events. Thus, on the election day we expect thefollowing sequence of timed events:

1. Session Start, Zero Totals Report (before the pollsopen).

2. Ballot Count Start (after the polls open).

3. Any number of override events (while the polls areopen).

4. Ender Card, Ballot Count End, Totals Report (whenthe polls close).

3.4 Automating the Event Log AnalysisTo automate the event log analysis, we created a tool thattakes, as input, both the event log and a set of timed rulesderived from the finite state machine presented above.The tool parses the event log string according to theserules to determine if the log represents an “expected” se-quence of timed events.

9

Figure 9: Set for Election with Zero Counters

Figure 10: Set for Election with Non-Zero Counters

Recall that the rules define a time-annotated state ma-chine that specifies the language of expected event se-quences. Certain transitions are enabled or disabled as afunction of the timestamps. For example, test electionscan take place in the dates before the election, but notduring the election or after the election. So we use thetimed rules to bind the normally-prescribed election pro-cess and the AV-OS state diagram. If the event log de-viates from what is considered expected, it is flagged as“irregular” and we further investigate its inconsistencies.

Table 2 shows three examples containing excerpt fromelection day event logs. Excerpt A shows an expectedsequence. Excerpt B is an irregular sequence that wasflagged because it has a voting terminal reset during theelection, at 11:20 am. Excerpt C is another irregular se-quence that was flagged both because it had a terminalreset at 13:17, but also because the election was neverended. From the inspection of the event log, we can seethat the voting terminal had power issues (consequentlythe poll officials had to switch to the back-up terminal).

4 AV-OS Event Log Defects/Deficiencies

This section describes one defect and two deficiencies ofthe event logging feature of the AV-OS voting terminal.Our findings are based on repeated and intensive teststhat examined numerous event log reports. The defectappears to be related to an undetected overflow conditionwhere the logging module of the AV-OS partially over-writes some entries of the log resulting in erroneous logprintouts. The two deficiencies stem from design weak-nesses of the logging module that fails to record certainevents and prevent any forensic analysis of the log to un-ambiguously determine which sequence of events leadto the observable content of the log. Our findings un-derscore the need for a complete analysis of the eventlogging module of the AV-OS.

4.1 Printing an Overflowed Event Log

The event log printed by the AV-OS consists of a se-quence of entries. Of these there are two types: a) actionentries and b) date entries. An action entry is a triple(s, n, t) where s is the sequence number (starting at 1),

10

Figure 11: Print Totals Report

Figure 12: Election Closed state

A. Expected Events B. Flagged Events C. Flagged Events06:45 SESSION START 06:45 SESSION START 05:17 SESSION START11-04-08 11-04-08 11-04-0806:51 ZERO TOT REPORT 06:49 ZERO TOT REPORT 05:25 ZERO TOT REPORT07:05 BAL COUNT START 07:02 BAL COUNT START 06:01 BAL COUNT START21:07 ENDER CARD 11:20 SESSION START 08:10 POWER FAIL21:07 BAL COUNT END 11-04-08 10:47 POWER FAIL21:11 TOTALS REPORT 11:20 COUNT RESTARTED 13:17 SESSION START

11:21 BAL COUNT START 11-04-0821:03 ENDER CARD 13:17 COUNT RESTARTED21:03 BAL COUNT END 13:20 BAL COUNT START21:13 TOTALS REPORT 14:44 POWER FAIL

Table 2: Examples of expected (A) and flagged (B and C) event logs

n is the name of the action, and t is the time at whichit occurred. The action entries SESSION START and INI-TIALIZED are always followed by a date entry. That dateentry records the date of the preceding action in the for-mat DD-MM-YY.

During an election, the internal log will often consistof several dozen entries recording a trace of the interac-tions of the terminal with its end-users. When the votingterminal is audited, one can request a printout of the in-ternal log.

A natural question regarding the robustness of the log-ging module is how it behaves when subject to a largenumber of (perfectly legal) interactions, resulting in anunusually lengthy log. Such an experiment reveals thatthe internal log is a fixed-size circular buffer of 512records. When more than 512 interactions occur dur-ing an election process, the newest entries overwrite theoldest ones. Despite this, one might expect the systemto faithfully report the portion of history reflected in thelog. That is, one might expect that with, say, 522 entries,the newest 10 of which have overwritten the oldest 10,

11

the printed log would indicate the following sequence:

(11, n11, t11), (12, n12, t12), . . . , (512, n512, t512),

(513, n513, t513), . . . , (522, n522, t522)

However, in the AV-OS, this is not the case. The printoutinstead shows:

(1, n11, t11), (2, n12, t12), . . . , (502, n512, t512), . . . ,

(512, n522, t522), (513, n11, t11), . . . , (522, n22, t22)

In essence, it appears that the printing module does notnotice the wrap-around and duplicates the entries fromthe prefix at the end. The immediate side-effect is thatthe paper printout reports action events with erroneoussequence numbers, out of order and duplicates some ofthem.

Perhaps even more surprisingly, we observed that ifsufficiently many (i.e., 512) pure action entries are cre-ated, the printed log becomes valid again. Namely, theentries are printed in their true order and the artificial du-plication effect disappears.

This observation suggests that, indeed, the loggingmodule detects overflows and behaves like a circularbuffer once it sees sufficiently many action entries. Itfurther suggests that action and date entries are recordedseparately in the internal log and each consume one slotin the circular buffer. This is not taken into account foroverflow detection. For instance, consider the situationwhere the log has 20 date entries and 500 action entries,for a total of 520 entries. The size of the buffer is 512,which results in an overflow of 8 entries. The newest 8entries overwrite the 8 oldest entries of the log. Howeverthe overflow remains undetected until the number of ac-tion entries exceeds 512, and we would thus observe theirregular behavior described above in the event log print-out. So if, for example, 1 date entry and 7 action entriesare overwritten the printed event log will show:

(1, n8, t8), (2, n9, t9), . . . , (493, n500, t500),

(494, n8, t8), . . . , (500, n14, t14)

In this sequence, the triples are associated with the ac-tion entries. SESSION START entries are followed by adate entry. If 13 more action entries are recorded in thelog, for a total of 533 entries (20 date entries and 513action entries), the overflow will be detected by the AV-OS. Let 1 date entry and 20 action entries be overwritten.The printout of the log will show the following.

(21, n21, t21), (22, n22, t22), . . . , (513, n513, t513)

Effects: This defect can potentially limit the auditingcapability to unambiguously determine the sequence ofevents that took place during the election and in fact al-lows a malicious manipulation of the log history. How-ever, it should be clear that this weakness only manifests

itself when a fairly large number of events are logged, apattern which, in itself, can be an unusual state of affairswhich can be easily detected. In any case, the processthat led to the automation of the audit log was instru-mental in uncovering a potentially serious defect in thelogging module that could be maliciously exploited.

4.2 “Totals Report” Recording Deficiency

At the end of the election day the officials must closethe elections and print the report with the results. Theprinting of this report is logged in the event log as theTOTALS REPORT action (Table 1). Prior election auditsindicate that, sometimes, the event does not appear on theevent log. Further investigation of the sequence of eventsthat might follow the election closing reveals that severaldistinct sequences of interactions between the machineand its operator can lead to the same log trace. Specifi-cally, we noticed that once we close the elections the ma-chine proceeds to the printing of a copy of the totals re-port. Once the printout is complete the machine asks theoperator whether s/he wishes another copy (as a yes/noquestion). If the operator replies negatively the machineproceeds and logs the action TOTALS REPORT and thenprompts for a restart. If, however, the operator replies af-firmatively (or does nothing at all) then the event is neverrecorded in the event log. Furthermore, while the ter-minal provides the capability of printing multiple copiesof the election results, at most one entry for the printingaction(s) is recorded in the event log.

Effects: No trace of printing the elections results maytrigger a controversy, questioning the authenticity of thetotals report, given than the event log does not confirmthe printing. Moreover the inability to identify the num-ber of reports printed may affect the auditing and elec-toral process. Note also that this behavior could beexploited to obtain a partial tally in the middle of theelection by closing the elections, printing a totals re-port, never acquiescing to the subsequent yes/no ques-tion, proceeding to power-cycle the terminal and then re-suming the election. Although the event log would con-tain a (benign) power-cycle sequence of events, there willbe no record that a partial tally was produced.

More fundamentally, this constitutes a deficiency inthe sense that there is no one-to-one mapping betweenthe actual sequence of user actions and events in the log.Ideal logging should record every action and not limititself to successfully completed actions or ‘at-most-once’logging of repeated actions.

12

4.3 Date Recording DeficiencyThe second deficiency directly relates to how date andtime is recorded in the logged events. Recall that eachaction entry is a triple (s, n, t), where t is the time atwhich the named event n occurred. The time has theform HH:MIN where the hour field ranges from 0 to 23.As indicated before, the action entries INITIALIZE andSESSION START are always followed by a date entry.These are the only action entries that cause date entriesto be recorded in the event log. The INITIALIZE actionentry is logged only when the machine is programmed.The SESSION START action entry is logged whenever thevoting terminal is power-cycled. To establish the timeand date of an event one must obtain the date from thelast preceding SESSION START event. Clearly, if the vot-ing terminal is left turned on for more than 24 hours thismethod fails to yield the correct time stamp.

Effects: Under the current procedures established inConnecticut, this deficiency cannot be exploited for anattack. However, if the equipment is used with relaxedprocedures where a district may upload their results toa central tabulator and avoid printing the actual electionresults, this may present a threat. Indeed, districts maykeep their voting terminals on after the election, allowfor more votes to be cast, then close the elections thenext day at the expected time. In this case no audit proce-dure of the event log will be effective. On the other hand,printing the totals report enables the officials to check theclose date on the printout.

5 Results of the Post-Election Analysis ofEvent Logs

Following the November 2008 election in Connecticut,we performed the analysis of the event logs collectedfrom 279 AV-OS voting terminals used in the election.Additionally, 142 event logs were collected from the (defacto) back-up AV-OS voting terminals that were notused in the election. Thus in total 421 event logs wereexamined with our automated log analysis tool. The 279collected event logs represent a random sample of over30% of all precincts, thus the audit is broad enough todraw meaningful conclusions. (See our earlier work [2]on how the event logs are obtained from the voting ter-minals.) In this section we summarize our findings.

• The majority of event logs, 314 out of 421 appearto contain the expected sequences of events (see [1]for the procedures followed by poll workers in Con-necticut).

• 15 event logs (3.6%) had more than 10 SES-SION START events. This refers to voting termi-

nal restarts. Some event logs had as many as fourrestarts in one minute.

Among these event logs, 10 logs (2.4%) addition-ally had a COUNT RESTARTED event during theelection day. This indicates that in some precinctsthere were difficulties with the terminals, perhapsballot jams. It is suggested that records be madeat the precincts in all cases where the terminals arerestarted during an election to help diagnose anyproblems in the future.

• 41 event logs (9.7%) contained card duplicationevents with 5 logs indicating more than one dupli-cation. Memory cards can be duplicated using a du-plication procedure of the AV-OS terminal. Thisfunctionality is available in supervisor mode. Thereshould be no reason to duplicate cards in Connecti-cut, and procedures clearly indicate so. One possi-bility is that duplication was performed when cardscontaining no valid data were discovered duringtesting. It is strongly recommended that duplica-tion should not be performed at precincts and thatall improperly programmed cards are reported to theSecretary of the State as soon as this is discovered.

• 29 event logs (6.9%) had a ZERO TOTALS REPORTprinted before the date of the election. This can hap-pen if one starts a voting terminal before the date ofthe election after setting it for elections.

• 24 event logs (5.7%) indicated that the corre-sponding memory cards were programmed from10/27/2008 to 10/30/2008. While this is not neces-sarily problematic in the election, these cards werenot subject to our pre-election audit that includedonly the cards programmed until 10/26/2008.

• 2 event logs had an additional ZERO TOTALS RE-PORT event during the election day. This happens ifone restarts the terminal after finishing printing thezero totals report.

• 1 event log had ELECTION CLOSE event at 22:08.This stands out from the rest of the cards, however,it is not necessarily problematic as voters waiting inline are still allowed to vote after doors are closed.

• 6 event logs had the PREP FOR ELECTION event onthe day of the election. Such event should havetaken place some days before the election and noton the election day. This suggests that pre-electiontesting procedures were not followed.

• 4 event logs had a MEMORY CARD RESET event.Resetting the cards clears the counters after the vot-ing terminal is placed in the election mode. All 4

13

event logs indicate that the reset was done before theelection date, thus no actual votes were lost. Nev-ertheless, this indicates a deviation from standardprocedures.

• 1 event log had an UPLOAD STARTED event. Thisindicates deviation from standard procedures (ap-parently the upload was attempted, yet there was noofficial receiving system). The corresponding cardwas not used in the election and otherwise the logappears normal.

• 2 event logs had test elections on 10/31/08 and 1event log showed a test election on 11/03/08. Weexpected test elections to happen earlier.

• 1 event log has a test election on 11/26/08 and anelection executed on 12/04/08. This suggests thatthe clock of the voting terminal was 1 month ahead.

None of the observations made in our analysis of theaudit logs indicate a security problem or malicious in-tent. However, it appears that proper procedures are notfollowed uniformly. These results have been communi-cated to the Connecticut Secretary of the State, and as aresult there will be amplification of election policies andprocedures.

6 Conclusions

We developed an automated tool for analyzing event logsof the Premier AccuVote Optical Scan (AV-OS) votingterminal. We have used the tool to analyze the event logsfrom over 400 memory cards during the post-election au-dit of the November 2008 elections in Connecticut. Thisaudit covered more than 30% of the precincts in the state.

The analysis of the event logs shows that there aresome deviations from the prescribed and expected useof the optical scan terminals. In particular, we noted that9.7% of the event logs contained memory card duplica-tion events. The Connecticut Secretary of the State in-structions to municipalities clearly do not allow for cardsto be duplicated. Additionally, several event logs con-tained deviations from the expected relative timing ofthe events (although these deviations are apparently non-malicious). Other deviations included restart events (ofbenign nature) and events that indicated clearing of elec-tion counters (fortunately in all cases before the elec-tion).

The time-annotated rules developed for our event loganalysis tool are specific to the AV-OS terminals and forthe election procedures established in Connecticut. How-ever, our approach is general and can be used to automatethe analysis of logs produced by other voting machines

under procedures specific to other states. This would re-quire that (a) the rules are revised to reflect the eventsof another voting machine and the election procedures ofanother state, and that (b) logs are converted to an ap-propriate .xml format. Moreover, for the states that up-load results for central tabulation, it should be possibleto automate the extraction of the logs from the tabulationsystem and analyze the extracted logs.

Having performed and completed the post-election au-dit after the November 2008 Elections, we believe thatvoting terminals need to be periodically audited and thatthis is crucial in providing valuable information neces-sary to ensure the integrity of our electoral system. Ourautomated event log analysis did not reveal an immediatesecurity concern or malicious intent. Nevertheless, ourin-depth event log experimentation revealed a defect andtwo deficiencies of the AV-OS logging subsystem. Fur-thermore, the collected information from the actual elec-tion indicates that the proper procedures were not uni-formly followed. We conclude that, despite minor defi-ciencies, event logs of voting terminals can be an invalu-able auditing tool and we recommend that event log anal-ysis be made a part of any post-election audit. We intendto continue our work on automated event log analysis andwe plan to investigate the design of logging capabilitiesthat would be rich enough to substantially increase theintegrity of elections using them.

References[1] A training guide for connecticut poll workers. Premier Elec-

tion Solutions http://www.sots.ct.gov/sots/lib/sots/electionservices/training_info/ct_poll_worker_manual.pdf, 2008.

[2] DAVTYAN, S., KENTROS, S., KIAYIAS, A., MICHEL, L.,NICOLAOU, N. C., RUSSELL, A., SEE, A., SHASHIDHAR, N.,AND SHVARTSMAN, A. A. Pre-election testing and post-electionaudit of optical scan voting terminal memory cards. In Proceed-ings of the 2008 USENIX/ACCURATE Electronic Voting Work-shop (EVT 08), July 28-29, 2008, San Jose, CA, USA (2008).

[3] DAVTYAN, S., KENTROS, S., KIAYIAS, A., MICHEL, L.,NICOLAOU, N. C., RUSSELL, A., SEE, A., SHASHIDHAR, N.,AND SHVARTSMAN, A. A. Taking total control of voting sys-tems: Firmware manipulations on an optical scan voting terminal.In Proceedings of the 24th Annual ACM Symposium on AppliedComputing (SAC 09) (2009).

[4] FELDMAN, A. J., HALDERMAN, J. A., AND FELTEN, E. W.Security analysis of the Diebold AccuVote-TS voting machine.http://www.usenix.org/event/evt07/tech/full_papers/feldman/feldman.pdf, 13 September2006.

[5] Help America Vote Act. http://www.fec.gov/hava/law_ext.txt.

[6] HURSTI, H. Critical security issues with Diebold opti-cal scan design. http://www.blackboxvoting.org/BBVreport.pdf, 4 July 2005.

[7] HURSTI, H. Diebold TSx evaluation. Black Box Voting Project,http://www.blackboxvoting.org/BBVtsxstudy.pdf, 11 May 2006.

14

[8] KIAYIAS, A., MICHEL, L., RUSSELL, A., SHASHIDAR, N.,SEE, A., AND SHVARTSMAN, A. An authentication and ballotlayout attack against an optical scan voting terminal. In Proceed-ings of the USENIX/ACCURATE Electronic Voting TechnologyWorkshop (EVT 07) (August 2007).

[9] KIAYIAS, A., MICHEL, L., RUSSELL, A., SHASHIDHAR, N.,SEE, A., SHVARTSMAN, A. A., AND DAVTYAN, S. Tamperingwith special purpose trusted computing devices: A case study inoptical scan e-voting. In Proceedings of the 23rd Annual Com-puter Security Applications Conference (ACSAC 2007), Decem-ber 10-14, 2007, Miami Beach, Florida, USA (2007), pp. 30–39.

[10] KOHNO, T., STUBBLEFIELD, A., RUBIN, A. D., AND WAL-LACH, D. S. Analysis of an electronic voting system. In Pro-ceedings of the IEEE Symposium on Security and Privacy (2004),pp. 27–.

[11] NORDEN, L. The machinery of democracy: Protecting electionsin an electronic world, 2005. Brennan Center Task Force on Vot-ing System Security, http://www.brennancenter.org/page/-/d/download_file_36343.pdf.

[12] PAMELA SMITH, VERIFIED VOTING.ORG. Written tes-timony before the Committee on House Administration,Subcommittee on Elections U.S. House of Represen-tatives. http://electionaudits.org/files/PamelaSmithTestimonyFinal_2007mar20.pdf,20 March 2007.

[13] VORA, P. L., ADIDA, B., BUCHOLZ, R., CHAUM, D., DILL,D. L., JEFFERSON, D., JONES, D. W., LATTIN, W., RUBIN,A. D., SHAMOS, M. I., AND YUNG, M. Evaluation of votingsystems. Commun. ACM 47, 11 (2004), 144.

[14] WAGNER, D., JEFFERSON, D., AND BISHOP, M. Security anal-ysis of the Diebold AccuBasic interpreter. Voting Systems Tech-nology Assessment Advisory Board, University of California,Berkeley, 14 February 2006.

[15] WERTHEIMER, M. A. Trusted agent report DieboldAccuVote-TS voting system. RABA Innovative Solution Cell,http://people.csail.mit.edu/rivest/voting/reports/2004-01-20%20RABA%20evaluation%20of%20Diebold%20AccuVote.pdf, January 2004.

15