Home

Certification Test Report - Florida Department of State

1. tassa Ie qm E SOURCE REVIEW E CONTINUOUS IMPROVEMENT amp RECOMMENDATIONS eeeeeee eene nennen 42 CONCLUSION s 43 CERTIFICATION OF EVS RELEASE 4 5 0 0 VERSION 4 VOTING SYSTEM cccccecessscececececcsssesescececeseeusneececeseeeausneeceeeeeaeenanens 43 PROVISIONAL APPROVAL OF EXPRESSVOTE BALLOT MARKING 43 hdd 45 hid pslmvesi rc APPENDIX B APPLICATION TIMELINE TABLE FIGURES INDEX scere peer hee Rennen erae Paese deo Ir gi HPR Q Page 2 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 Executive Summary Elections Systems amp Software LLC ES amp S submitted an application requesting Florida certification of the EVS Release 4 5 0 0 Version 4 voting system ES amp S s request includes various optimization and performance enhancements of previously certified systems as well as the introduction of the ExpressVote marking device The current Florida certified voting system includes an election management system kn
2. tenens 15 CONDUCT OF TESTS FINDINGS ses rais een otov aea Fe Seo a vin a sae Seo leen a SUR ER Y SE EE eS UN aS ES TRE ER EE NUR Sa 16 SYSTEMS SETUP Se CONFIGURATION Ee ERE RUF Ve ERR TR 16 preis e YU 16 FU NGIHONAESYSTEMCUAUDIET epe neenon e neces 16 Voting Equipment Menus Administrative and Diagnostic Reports eese enean 16 MOCK cR 9S Precinct Tabulator Mass Ballot Accessibility Sound Pressure amp deret Accessibility Clearance Requirements Contest Recounts esses enne Regional Results Application Folded Balo Yer hdeet qst Syst m tacavaeeiauaeeeevateas ten X Q EXPOS SVOTE ut ears coaches ExpressPass Printer Bar eee
3. Voting System Qualification Test Report Elections Systems amp Software LLC EVS Release 4 5 0 0 Version 4 April 2015 FLORIDA DIVISION OF ELECTIONS Florida Department of State Division of Elections R A Gray Building Room 316 500 S Bronough Street Tallahassee FL 32399 0250 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 Contents EXECUTIVE SUMMARY rosc 3 INTRODUCTION p 5 BACKGROUND gt M n 5 SYSTEM M 5 COMPONENT VERSION LIST ems Ec 7 SOFTWARE System Software HARDWARE tero eet Pies neue Election Management System Hardware Precinct Count Systems is s a Precinct Voter Assistance Devices Aoda Central Absentee asa teas eas Voting System HAAMO E CUR PER Va VAPEUR COMPONENTS UNDER etnneee tese
4. 11 1 1 0 0 Jetpack 4G LTE Netgear Zing 4G LTE USB551L Netgear 341U Netgear 340U Voting System Qualification Test Report ES amp S EVS Release 4 5 0 0 Version 4 Manufacturer Symantec Adobe IPSwitch ES amp S Microsoft WSUS Symantec Adobe ES amp S Verizon Sprint AT amp T Verizon Sprint AT amp T ExpressPass Optional ExpressPass Workstation Description Anti Virus PDF reader for software help guides Secure FTP server Processes tabulated results from DS200 DS850 Regional Results Optional Regional Results Client for network environment Operating System Offline operating system updates for security etc Anti Virus PDF reader to access the help guide ES amp S Software Application Optional Wireless Hotspot for regional results transmission Optional Broadband Modems for regional results transmission Windows Operating System Microsoft Offline Updates Symantec Endpoint Protection Adobe Reader XI ES amp S ExpressPass ExpressPass Printer Windows 7 Professional SP1 8 6 Small Business Edition 2013 XI 11 1 0 0 0 Model 402 Microsoft WSUS Symantec Adobe ES amp S Microcom Operating System Offline operating system updates for security etc Anti Virus PDF reader to access the help guide ES amp S Software Application ExpressPass printer to print activation cards Page 9 of 49 Florida De
5. AutoCast function that allows the user to have the card automatically placed into the secure AutoCast box for later tabulation by election officials using a DS200 or DS850 BVSC observed the ExpressVote in several test scenarios to examine the capabilities of this new device Mass Marking Examination ExpressVote BVSC conducted a mass marking test to simulate a high number of ballots to be processed by the ExpressVote BVSC used the 2012 Escambia County Primary Election a single card election to create a test deck by marking 257 activation cards using the ExpressVote BVSC used blank cards as well as cards with activation codes pre printed using the ExpressPass printer BVSC cast the test deck in both a DS200 and a DS850 then compared the two tabulator results 25 Florida ES amp S Voting System 4 5 0 0 v4 System Functionality Description pg 90 Page 29 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Findings BVSC found no anomalies Multi Card Ballot ExpressVote Just as a paper ballot may consist of more than one sheet of paper so the ExpressVote vote summary may summarize votes on more than one vote summary card The vendor included this feature as a modification to the original application for certification in order to enhance vote summary space BVSC examined the voting system s capability of handling an ExpressVote multi card ballot
6. Document Version 1 0 Firmware Version 2 8 Feb 6 2015 pg 56 8 The vendor s application for certification lists stock headphones in the Component Version List The specifies the following headphones AVID Airline Products model number 1A6FVO60CBK32ST ExpressVote Operator s Guide Firmware Version 1 2 Document Version 2 0 pg 24 The vendor supplied the following AVID educational headphones unmarked ITU T P 50 ITU T is the telecommunication standardization sector of the ITU which is the International Telecommunication Union ITU is a United Nations specialized agency for information and communication technologies The P 50 represents one of their P Series objective transmission standards measures used for testing the transmission quality of artificial voices Page 20 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 The test equipment included a Type IEC 318 Br el amp Kjaer Artificial Ear Br el amp Kjaer Free field inch microphone Type 4189 and Br el amp Kjaer model 2250 A analyzer all of which were calibrated AutoMARK The AutoMARK was tested with the headphones provided as part of the Technical Data Package TDP Findings BVSC found that the AutoMARK complies with the applicable statute The results of the sound pressure level tests for the AutoMARK are in the table below Tabl
7. 1 1 Feb 25 2015 pg 31 Page 35 of 49 Florida Department of State Division of Elections Voting System Qualification Test Report ES amp S EVS Release 4 5 0 0 Version 4 Figure 16 ExpressVote precinct split and ballot style selection screens and confirmation screen with blank vote summary card insertion 2D Precinct Selection Search Enter name or number and touch Search To see all available choices just touch Search 4 Precinct PRECINCT 837 SPLIT 1 Select a ballot E DEM PRECINCT 837 SPLIT 1 36 REP PRECINCT 837 SPLIT 1 5 GRN PRECINCT 837 SPLIT 1 76 PRECINCT 837 SPLIT 1 Precinct Selection Results Select a precinct PRECINCT 837 SPLIT 0 RECINCT 837 SPLIT 1 Confirm Selections Verify that your selections are correct Precinct PRECINCT 837 SPLIT 1 16 DEM PRECINCT 837 SPLIT 1 Step firm your selections and display the ballot touch uvate o make a different selection touch Search 4 The Vote for x contest instruction did not match the vote for setting across all voting system components During testing BVSC found that there was a difference in the displayed vote for number among various voting system components The general election definition was coded with a Vote for 2 setting in a particular contest and the tabulators correctly tallied the contest as a Vote or 2 Figure 17 Page 36 of 49 Florida Department of State Voting S
8. 5 0 0 Version 4 which was submitted on February 10 2015 posed no significant safety security or operational risks Continuous Improvement amp Recommendations During the extensive period of testing and modifications BVSC identified several areas affecting usability and functionality that warrant enhanced documentation procedures poll worker and staff training or refinements to the system procedures or peripherals BVSC makes the following recommendations for the proposed certification of the EVS Release 4 5 0 0 Version 4 voting system and or provisional approval or extension of the provisional approval of the ExpressVote component to enhance the use of this voting system in its current version if certified and to improve the voting system including ExpressVote in this and future releases 1 In order for certification of the EVS Release 4 5 0 0 Version 4 voting system to be granted or for provisional approval of the ExpressVote component to be granted the vendor should revise the user manual and other documentation a To specify more clearly how a user may test the functionality of the DS200 accommodating the autumn Daylight Saving time change b To direct the user to generate voter instructions and other ballot style content from the source fields rather than to enter them manually in the style sheets specify more clearly the programming of Style Sheets with multiple ExpressVote cards d To advise Expres
9. ExpressPass printer only prints one vote summary card at a time g Usecare when inserting vote summary cards into the ExpressPass printer Page 44 of 49 Florida Department of State Division of Elections Appendices Appendix A Acronyms Voting System Qualification Test Report ES amp S EVS Release 4 5 0 0 Version 4 ADA Americans with Disabilities Act BVSC Bureau of Voting Systems Certification CF Compact Flash memory cards COTS Commercial off the Shelf software hardware dB Decibels EAC U S Elections Assistance Commission ECO Engineering Change Order EMS Election Management System ERM Election Reporting Module ES amp S Elections Systems amp Software LLC EVS ElectionWare Voting System F S Florida Statutes FVSS Florida Voting Systems Standards GB Gigabytes HAVA Help America Vote Act HDD High Density Drive LAN Local Area Network L amp A Logic and Accuracy voting system test MB Megabytes PPP Presidential Preference Primary election TDP Technical Data Package USB Universal Serial Bus VAT Voter Assist Terminal VVSG Voluntary Voting Systems Guidelines VSTL Voting Systems Test Laboratory XML eXtensible Markup Language Page 45 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Appendix B Application Timeline Table ACTION SYST
10. Printer e Universal Power Supply APC1500 e USB Memory Stick Media for DS850 Page 13 of 49 Florida Department of State Division of Elections Voting System Qualification Test Report ES amp S EVS Release 4 5 0 0 Version 4 o Delkin USB 512MB 1GB 2GB AGB 8GB e Compact Flash Memory Card o Delkin Compact Flash 1GB In addition to the above components ES amp S is seeking approval for an upgrade to the optional Toolbox support utility e Toolbox v 2 0 0 0 Toolbox v 2 0 0 0 is a software upgrade to the previously approved Toolbox v 1 0 0 0 zj version Toolbox is a software utility used in conjunction with ElectionWare software that jurisdictions may utilize to generate election audio files and test decks Several performance enhancements were made to the approved 1 0 0 0 zj version Voting System Hardware The hardware components of the voting system are described in the table below Table 3 Voting system hardware components Product Version Manufacturer Description DS200 System Firmware 2 11 0 3 ES amp S Application Firmware for precinct based vote tabulation Scanner Firmware 3 0 0 0 ES amp S Board level Firmware Power Management 1 2 10 0 ES amp S Board level Firmware Firmware System Hardware 1 2 1 2 3 0 1 3 ES amp S DS200 Voting System Hardware Compact Flash Memory N A Delkin Supported sizes include 1 GB Card N A Delkin Supported sizes include USB dba 5tick 12 13 14 ES amp S 512MB 1GB 2GB 4GB 8G
11. T Verizon 26 Figure 9 Connecting the Regional Reporting Site to the AT amp T Hotspot via Wi Fi technology 27 Figure 10 Contest selection for ExpressVote ballot styles esses 31 Figure 11 Contest on ExpressVote Card 1 and Card 2 figure shows ElectionWare ballot style view 32 Figure 12 Double contest on ExpressVote ballot and counted in election results 32 Figure 13 ExpressVote onscreen display full contest candidate 33 Figure 14 Contest titles and candidates paper ballot vs ExpressVote summary 34 Figure 15 ExpressVote inconsistent card order 35 Figure 16 ExpressVote precinct split and ballot style selection screens and confirmation screen with blank vote summary card iNSELtiON cccccesssceessceceessssececsesaececessseceesessececsesaeeeeseaaececsesaeeecseaaeeeeseaaes 36 Figure 17 Contest coded for Vote for 2 05850 tabulated Vote for 2 37 Figure 18 Paper ballot instruction Vote for ExpressVote Vote for no more thanTwo 37 Figure 19 ExpressPass on a computer with ExpressPass thermal printer 39 Figure 20 Voter activation card sho
12. The vendor should remedy this risk for future extension of the provisional approval for the ExpressVote if granted or in a future system release Page 32 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Vote Summary Card Capacity ExpressVote The ExpressVote vote summary card was examined in terms of its capacity It is important to note that although ElectionWare allows a user to set the font size for the ExpressVote vote summary card to smaller sizes only the large font size setting corresponding to a 10 point font or greater is allowed in the state of Florida 2 Furthermore contest names on a ballot display card cannot be abbreviated or truncated Findings The ExpressVote onscreen displays like a paper ballot That is the display shows the contest titles and all the candidate field names in their entirety Figure 13 Figure 13 ExpressVote onscreen display full contest candidate name However the fields referenced for printing on the ExpressVote summary card are not the same as those displayed on a paper ballot the accessible ballot preview or the accessible ballot onscreen displays The vote summary card displays a maximum of 38 characters per line for contest titles and candidate names when the font size is set to large Titles and names exceeding the character limit do not wrap to the next line so they are truncated to
13. be conditioned upon the vendor programming the following feature in any future release a user warning and or confirmation process whenever manual user created text is used in lieu of data fields to generate the ballot styles In the short term the certification should be conditioned on the vendor providing explicit procedures in its user manual and other documentation to help users of the system who Page 37 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 choose to manually enter voter instructions and other ballot style content in the style sheets to verify their choices During the EVS Release 4 5 0 0 ExpressVote marking device examinations BVSC also identified the following issues that required vendor response e Print button fatal error During a session in which the audio tactile keypad was engaged BVSC found that pressing the print button on the screen caused the ExpressVote to present a fatal error o The vendor was notified and responded with a modification to the application for certification due to a required firmware update for the ExpressVote BVSC re tested this function and found no further anomalies e Pause button fatal error BVSC encountered a software bug the ExpressVote that produced a fatal error when the pause button on the audio tactile keypad was pressed o ES amp S responded with a new version of the Express
14. summary Findings 1 Possibility of voting the same contests more than once when using blank ExpressVote card With the ExpressVote the insertion of each blank card initiates a new voting session In a multiple card election a risk exists that a voter can vote the same contests more than once For example A voter is given two blank cards to use as Vote Summary Card 1 and Vote Summary Card 2 which represent the complete ballot After the appropriate precinct is selected on the ExpressVote the ExpressVote displays the two ballot styles that are valid for that precinct Voter selects Vote Summary Card 1 as his first set of contests to vote and proceeds to make vote selections and verify them Voter then AutoCasts his vote summary card Now voter is ready to make selections for the remaining contests on Vote Summary Card 2 however there is nothing to indicate to the voter or to prevent the voter from inserting his remaining blank card and selecting Vote Summary Card 1 ballot style again ES amp S recommends that only the ExpressPass printer be used for multi card elections With a pre printed header bar code on each vote summary card the voter can only vote the ballot style identified by the header bar code The ExpressPass header bar code also contains the voter s precinct which eliminates confusion and mistyping at the beginning of the ExpressVote voting session The vendor must enhance the ExpressVote blank card multi card voting
15. voting system under test matched the specifications described in the application and the TDP documentation Findings BVSC found no discrepancies with the setup of the ElectionWare voting system configurations Functional System Audit BVSC conducted a functional system audit to verify that all components of the voting system operate as described in the TDP Voting Equipment Menus Administrative and Diagnostic Reports BVSC performed a functional audit by testing all available menu options and administrative reports as well as systems functions in the course of testing Page 16 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Findings The system performed as indicated in the vendor s TDP and in accordance with FVSS Florida Statutes and Administrative Rules Mock Elections BVSC conducted mock elections incorporating multiple card ballots of varying ballot lengths 11 inch to 19 inch BVSC used four election types Presidential Preference Primary PPP municipal primary and general The tests included both hand marked and machine marked ballots as well as those cast via the accessible voting ballot marking machines Auto MARK and ExpressVote and used single card and multi card elections All activities simulating an election were conducted using all equipment from initial preparations and L amp A pre election activities through vot
16. 0 0 Version 4 voting system except for the ExpressVote component provided the vendor makes beforehand necessary changes to its written Page 3 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 documentation to reflect BVSC recommendations for greater specificity clarity and guidance regarding use of the system 2 Provisional approval of the ExpressVote ballot marking device for use in any election conducted through May 1 2017 with the ExpressVote ballot marking device to be used solely by persons with disabilities The approval should be subject to the following conditions a The vendor should make the necessary changes to its written documentation to reflect BVSC recommendations for greater specificity clarity and guidance regarding use of the system b The use of the ExpressVote ballot marking device if approved is also subject before use in any election to future revisions to Rule 1S 2 032 F A C to provide standards for the accessible audio visual onscreen display of the ballot and the printed vote summary card in accordance with section 101 5608 3 Florida Statutes c BVSC does not recommend an extension of this provisional approval if granted or recommend the approval of any future release of this ExpressVote ballot marking device unless the issues in the Continuous Improvements Recommendations section of this report are addressed or ar
17. 200 DS850 and ExpressVote use Delkin brand ES amp S qualified USB flash drives Page 6 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Component Version List The component version list describes in detail the components of the voting system under test Software The software applications that comprise the voting system are listed below e Event Log Service ELS version 1 5 4 0 e Removable Media Service RMS version 1 4 4 0 e ElectionWare EW version 4 5 0 0 AutoMARK VAT Previewer version 1 8 5 0 e ExpressVote Previewer version 1 2 3 0 e Election Reporting Manager ERM version 8 10 0 0 e Optional Regional Results version 1 1 0 0 e Optional ExpressPass version 1 0 0 0 System Software The system s software requirements are in the table below Table 1 Election Management System software requirements Software Firmware Manufacturer Description Version EMS Workstation or Standalone Configuration Required for all EMS components Windows Operating Windows 7 Microsoft Operating System System Professional SP1 Microsoft Offline 8 6 WSUS Offline operating system Updates updates for security etc Symantec Endpoint Small Business Symantec Anti Virus Protection Edition 2013 Adobe Acrobat Standard XI 11 0 Adobe Ballot PDF generation RMCOBOL 12 06 Liant COBOL Runtime Removable Media 1 4 4 0 ES amp S Supports gracef
18. B Plastic Ballot Box With or without ES amp S Plastic Ballot receptacle Metal Ballot Box Diverter Metal ballot receptacle DS850 System Firmware 2 8 2 2 ES amp S Application Firmware for central based vote tabulation System Hardware 1 0 ES amp S DS850 Voting System Hardware Compact Flash Memory N A Delkin Supported sizes include 1 GB Card USB Memory Stick N A Delkin Supported sizes include 512MB 1GB 2GB 4GB 8GB Report Printer N A OKI B430dn Laser Report Printer OKI B431dn Laser Report Printer Audit Printer N A OKI Microline 420 Dot Matrix Printer Page 14 of 49 Florida Department of State Division of Elections Voting System Qualification Test Report ES amp S EVS Release 4 5 0 0 Version 4 USB Memory Stick ExpressVote Voting System System Firmware 1 2 3 0 ES amp S Application Firmware for ES amp S Voting System Input Output Board 1 0 0 0 ES amp S Board Level Firmware Scanner Printer Engine 1 0 0 0 ES amp S Board Level Firmware Board SPE Detachable Key Board 1 0 0 0 ES amp S Board Level Firmware DKB System Hardware 1 0 ES amp S Hardware revision for ExpressVote Supported sizes include 512MB 1GB 2GB 4GB 8GB Headphones Stock headphones ES amp S AutoMARK System Firmware 1 8 5 0 ES amp S Application Firmware for HAVA compliant ballot marking system Printer Engine Board PEB 1 70 ES amp S Board Level Firmware Scanner Interface Board 1 43 ES amp S Board Leve
19. EM VERSION DATE RECEIVED REQUESTED Original application EVS 4 5 0 0 Version 2 12 09 2013 ModHicaon 1 EVS 4 5 0 0 Version 2 02 07 2014 Request Requirement Modification 2 EVS 4 5 0 0 Version 2 04 08 2014 Request Requirement Modes sels EVS 4 5 0 0 Version 2 04 16 2014 Request Requirement Medication EVS 4 5 0 0 Version 2 04 18 2014 Request Requirement t3 EVS 4 5 0 0 Version 2 05 08 2014 Request Requirement Testing was suspended July November 2014 by mutual agreement Modification 6 EVS 4 5 0 0 Version 3 12 12 2014 Request Requirement Modification EVS 4 5 0 0 Version 4 02 06 2015 Request Requirement Page 46 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 Figures Index Figure 1 EXDressVOte iiie e e ec se cede Ree cc cg Seeds Ei ee nde le PER er RR CY bep EE e DE eve d s 6 Figure 2 AUtoMARK iicet terere e ee eve e coed ve ev ex P qe Lu x ER cade vae ex PH aed Leak 6 FIBUPE 3 DS2 m 6 Figure 4 DS850 eta tester te e e debs m Peer EE cage YER cede Se utr ive e and Re eee ete eden 6 Figure 5 ExpressVote undergoes testing with a force gauge enne enne 22 Figure 6 Express Vote voting booth ote ee Ei eee ecce eue edt Er uae eee E eu 23 Figure 7 ExpressVote AutoCast Table with and without privacy 24 Figure 8 Hotspots and Broadband Modems AT amp
20. RI 23 DISTRITO DEL TRIBUNAL DE E t to your correct choice CNG23 2 RE APELACIONES the oval next to y 23rd Congressional District DISTRIK TRIBINAL DAP L i PARA VOTAR LLENE ES Vote for One Vote por uno Vote pou youn Shall Judge Angel A Cortinas of the X Use un bol grafo de tinta negra o azul Si Karen Harrington REP 26 ord nent Court of Appeal be retained comete un error t chelo con una cruz el Mu valo junto a su selecci n correcta erse en su cargo al Juez s del Tribunal de Distrito POU VOTE RANPLI ANDEDAN OVAL LAN r Distro nen NET AKOTE SA W CHWAZI AN Cortinas S vi th plim nwa vem ble Siw ane nou F Ap l dwe fot er f yon kwa sou fot lan epi ova akote sa w chwazi an MOCK ELECTION FLORIDA 2 CARD EXPRESSVOTE ELECTION 82 17 2815 PRECINCT 881 8 EXPRESSVOTE CARD 1 PRE PRESIDENTE Y VICEPRESIDENTE Tt PREZIDAN AK VIS PREZIDAN MTT TIT UD EU e for por unoNote pou youn ULL LU M i IUE Mitt Romney oe LLL PRESIDENT VICE PRESIDENT FEDERAL O Barack Obama REP MITT ROMNEY UNITED STATES SENATOR FEDERAL ONLY REP CONNIE Els Thomas Robert Stevens OBJ 12 CNG23 REPRESENTATIVE u CONES 1 L LI u u u L u REP KAREN HARRINGTON Alden Lin
21. T JUDGE GROUP 10 COUNTY COURT JUDGE GROUP 11 Before Grouping Ballot Styles 7 COUNTY COURT JUDGE GROUP 12 Rotate Candidates 1 Home Position COUNTY COURT JUDGE GROUP 14 COUNTY COURT JUDGE GROUP 27 COUNTY COURT JUDGE GROUP 39 Ballot Style Information COUNTY COURT JUDGE GROUP 40 Ballot Style Name Prefix COUNTY COURT JUDGE GROUP 43 REP Card 1 V County Question Ballot Set Usage Ballot Set Used With ExpressVote Only v V increment Total Number of Ballots ElectionWare s ballot style preview shows the contest on both Card 1 and Card 2 Figure 11 It is important to note that the paper ballots do not have this problem because the contest selection is generated automatically for paper ballots Page 31 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Figure 11 Contest on ExpressVote Card 1 and Card 2 figure shows ElectionWare ballot style view Wi KONPLETMAN NU L OVAL NEXT PO ESTY QV SEV SEMAN PL M YO BAY LA Si ou Ry COUNTY COURT GROUP 40 CONDADO DE JUEZ DELA CORTE GRUPO KONTE Jj Tsbnal GWOUP 40 LI sera 2 COUNTY COURT JUDGE GROUP 43 CONDADO DE REZDELACORIE GRIFO COUNTY COURT JUDGE GROUP 43 CONVDODE Tribnal GWOUP 43 ll BVSC demonstrated that the double contest problem perpetuates through the voting and tallyi
22. Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Figure 7 ExpressVote AutoCast Table with and without privacy hood ExpressVote AutoCast Table Card Bin The dimensions of the table are as follows Table Area Dimension Top of table surface 33W X 25D X 31H Bottom of table surface 34W X 26D X 27H Leg span 37H X 29D Findings BVSC measured the minimum height of the voting controls and the minimum knee clearance opening of the ExpressVote table with the AutoCast box BVSC the found that the table complied with section 101 56062 m F S In lieu of testing for conformance to Florida Statutes BVSC accepts the VSTL test results on usability and accessibility for the ExpressVote voting booth 19 ExpressVote Standard Operating Procedure ver 1 0 Chap 3 20 The VSTL is NTS Huntsville AL Usability and Accessibility test performed in accordance with Volume 1 Section 3 of the EAC 2005 VVSG Report No T71379 01 01 Rev B June 20 2014 Page 24 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 Contest Recounts BVSC conducted a recount to verify compliance with section 102 141 7 F S and the FVSS BVSC selected one countywide race and one district wide race in the municipal election The recount was conducted using a DS200 precinct tabulator and a DS850 high speed tabulator BVSC also c
23. VS Release 4 5 0 0 Version 4 e Compact Flash Memory Card o SanDisk Compact Flash 512MB 1GB 2GB o Toshiba Compact Flash 512MB 1GB 2GB e San Disk Image Mate or similar compact flash interface for use with compact flash cards AutoMARK Central Absentee Count The central count or absentee count scanners available are the DS200 and the DS850 and include one or more of the following Digital Scan 200 DS200 e DS200 Hardware Revision 1 2 1 2 3 0 1 3 o 05200 Hardware v 1 2 3 0 is equivalent to HW 1 2 plus ECOs 1158 1160 1281 1346 1398 1400 Approved by BVSC on 7 31 13 e Firmware version 2 11 0 3 o Scanner Firmware v 3 0 0 0 o Power Management Firmware v 1 2 10 0 e Auxiliary equipment for 05200 o Plastic Ballot Box HW Revision 1 2 1 3 1 4 Plastic Ballot Bin HW v 1 4 is equivalent to HW 1 3 plus ECOs 1251 1266 1351 1388 Approved by BVSC on 7 25 13 and 7 31 13 o Steel Ballot Box With or without diverter o Plastic 05200 Tote Bin e USB Memory Device Media for DS200 o BDelkin USB 512MB 1GB 2GB 4GB 8GB e Compact Flash Memory Device Internal for 05200 o Delkin Compact Flash 1GB Digital Scan 850 DS850 Central Count Ballot Tabulator e DS850 Hardware Revision 1 0 e Firmware version 2 8 2 2 e COTS Printers o OKI B430dn Laser Report Printer Dedicated Report Printer o OKI B431dn Laser Report Printer Dedicated Report Printer o OKI Microline 420 Dot Matrix Printer Dedicated Audit Log
24. Vote firmware BVSC re tested this function and found no further anomalies e Vote target display separation During testing BVSC discovered that the yes and no referenda vote targets could be separated on the display screen The screen displayed the yes vote target at the bottom of one screen and the user would be required to scroll to the next screen to view the no vote target This is contrary to Uniform Primary and General Election Ballot Rule 1S 2 032 8 o ES amp S responded with a new software version that fixed this display issue BVSC re tested this function and found no further anomalies ExpressPass Printer Bar Codes ExpressPass is an optional on demand application that is used to print activation cards with bar codes containing precinct and ballot style identification for use with ExpressVote The stand alone application resides on a computer connected to an ExpressPass thermal printer The ExpressPass application functions in either stand alone mode or in a monitor mode where the application monitors requests from a voter registration system over a shared network folder The application is used to import an election definition from ElectionWare process requests for a given precinct then determine the voter s ballot style and print a ballot activation code bar code on the activation card using the ExpressVote thermal card printer 30 An activation card is an ExpressVote blank vote summary card that has the activati
25. anomalies and the system functioned as expected Regional Results Application The voting system under test is capable of receiving and accumulating results transmitted via regional reporting sites The Regional Results software resides on a computer with a defined modem capability In ElectionWare and the Regional Results Reporting software application each polling location is assigned to a reporting region The software application in the test was installed on a stand alone laptop Staff conducted the PPP election using three reporting groups Region 1 Region 2 and Region 3 Staff manually uploaded the results to the Regional Results Reporting software and then transmitted the results via wireless modem using an AT amp T Hotspot to the SFTP server 22 ElectionWare Volume I Administrator s Guide Document Version 1 0 Software Version 4 5 v4 February 7 2015 pg 92 Because the APN number will already be coded for the location of the end user of the system a Florida county will not have this issue The APN number was not coded before arrival for testing in Tallahassee 23 Regional reporting sites are sites located in a county or other election jurisdiction that receive manually uploaded results from nearby polling places They then electronically transmit those results to the SOE office This method allows a county to send results to the SOE office quicker than complete manual uploading and less expensively than the cost of deployin
26. as a distribution system to display scrolling results on screen and over the Intranet e The ballot marking equipment includes the ExpressVote and AutoMARK voter assist terminal VAT The tabulating voting equipment includes the DS200 and the DS850 3 See Appendix B Application Timeline Table Page 5 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Figure 1 ExpressVote The ExpressVote is a vote capture device with a verifiable paper record that is digitally scanned for tabulation This system combines paper based voting with touch screen technology and integrates assistive peripherals such as headphones tactile keypad sip and puff device or rocker paddle control switch The ExpressVote includes a mandatory vote summary screen that requires voters to confirm or revise selections prior to printing the summary of ballot selections using the internal thermal printer Once printed ES amp S ballot scanners process the vote summary card ES amp S has fully integrated the ExpressVote with the existing suite of ES amp S voting system products The AutoMARK Voter Assist Terminal VAT is a voter interface device that allows a voter to mark a blank preprinted paper ballot or assists a voter with contest selections via visual display audio or both The voter uses the AutoMARK to navigate the ballot utilizing the touchscreen physical keypad and assistive su
27. e purpose of the scanner sensitivity test is to observe the scanner s ability to read marks made by various types of marking instruments pens pencils highlighters etc The sensitivity test also demonstrates the scanner s ability to detect a marking when the vote target is not fully or properly marked such as O etc BVSC created the test decks by marking the first position on blank ballots with various marking instruments The test deck included two ballots for each marking instrument a baseline ballot with selected ovals fully marked 8 and a test ballot marked with a horizontal line 1 millimeter thick through the center of the target For testing the scanners ability to detect a variety of improperly marked targets BVSC marked the target using the vendor s approved pen VL Ballot Pen a BIC Grip Roller ball point pen 7mm Part 6100 These results were compared to a baseline of the same targets fully marked 49 using the same pen Findings BVSC observed that the precinct and central count scanners are able to detect a wide variety of marks made by several different marking instruments However the scanners consistently detected marks by marking instruments limited to softer grades of pencil 2B to 6B a black felt tip pen and the vendor s recommended pen VL Ballot Pen As such the vendor clearly documents in its user manual and other documentation the recommended marking device s System Clock BVSC
28. e 8 Sound pressure level test results AutoMARK Sound Pressure Level Test Results AutoMARK Average Maximum Average Minimum Gain dBA Intermediate Level Volume dBA Volume dBA dBA Right Headphone 111 20 61 20 76 20 Left Headphone ExpressVote The ExpressVote unit was tested with headphones provided as part of the TDP Findings BVSC found that the ExpressVote complies with the applicable statute The results of the sound pressure level tests for the ExpressVote are in the table below Table 9 Sound pressure level test results ExpressVote Sound Pressure Level Test Results AutoMARK Average Maximum Average Minimum Gain dBA Intermediate Level Volume dBA Volume dBA dBA Right Headphone 109 80 71 70 38 10 83 90 Left Headphone 109 20 71 20 38 00 83 30 Accessibility Force BVSC conducted the force test on the AutoMARK and the ExpressVote using multiple input methods for each the available touchscreen the rocker paddle switch and the audio tactile keypad ExpressVote only BVSC conducted these tests to determine compliance with section 101 56062 1 l F S which 19 IEC International Electrotechnical Commission IEC 318 is a measure used for ear simulators as defined in ITU T P Series standards 11 Br el amp Kjaer 2250 Analyzer A hand held analyzer and sound level meter that performs high precision measurement tasks in environmental occupational and industrial application areas 12 Must be
29. e no longer an issue 1 ES amp S has satisfactorily completed EVS Release 4 5 0 0 Version 4 manual updates documentation requirements ES amp S has satisfactorily completed manual updates documentation requirements as related to the ExpressVote ballot marking device Page 4 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Introduction Elections Systems amp Software LLC ES amp S submitted an application requesting Florida certification of the EVS Release 4 5 0 0 Version 4 voting system ES amp S s request includes various optimization and performance enhancements as well as the introduction of the ExpressVote marking device The ExpressVote is a voter interface device for voters with disabilities to be used at a polling location Background ES amp S s ElectionWare voting system first received certification in the state of Florida on February 9 2012 for EVS Release 5 0 0 0 Version 2 On September 25 2012 BVSC certified EVS Release 4 0 3 0 Version 2 which included enhancements to the precinct and high speed scanners of the voting system In January 2014 ES amp S s initial application for certification of EVS Release 4 5 0 0 Version 2 was deemed to be complete Through the subsequent review and testing process the vendor made changes to the scope of the application and necessary modifications to the system The application now in its sev
30. ecure FTP server for DS850 file transfer COBOL Runtime Supports graceful installation and removal of media Provides custom event log reports from the Windows Event Log Installs the Postgres database Network Results Server Optional Results Server for Modeming Support Windows Operating Server 2008 System R2 Standard Edition Microsoft Offline 8 6 Updates Symantec Endpoint Protection Small Business Microsoft WSUS Symantec Edition 2013 Cerberus FTP 6 0 7 1 Cerberus LLC Operating System Offline operating system updates for security etc Anti Virus Secure FTP server Network Results Client Optional Windows Operating Windows 7 System Microsoft Offline 8 6 Updates Professional Results Client for Modeming Microsoft SP1 WSUS Page 8 of 49 Support Operating System Offline operating system updates for security etc Florida Department of State Division of Elections Product Symantec Endpoint Protection IPSwitch WS FTP Election Reporting Manager ERM Windows Operating System Microsoft Offline Updates Symantec Endpoint Protection Adobe Reader XI Vendor Wireless Hot Spots Vendor Broadband Modems Adobe Acrobat Standard ES amp S Regional Results Software Firmware Version Small Business Edition 2013 XI 11 0 12 4 1 8 10 0 0 Windows 7 Professional SP1 8 6 Small Business Edition 2013 XI
31. enth iteration as EVS Release 4 5 0 0 Version 4 3 includes optimization and performance enhancements throughout the voting system System Overview This is a paper based voting system with an element for compliance with HAVA provisions for precinct voting The Florida certified voting system includes the Election Management System EMS a precinct scanner a central count scanner and precinct voter interface devices that can be used by a voter with a disability The EMS hardware platform is configured as either a stand alone or a server client configuration The stand alone configuration includes the election management system and the election results reporting manager whereas the server client configuration includes one or more workstations clients which interconnect with a server The system includes an option to upload election results wirelessly or using an analog landline modem The EMS software configuration includes e ElectionWare integrates the jurisdiction districts contests and candidate database as the main pre voting phase and post voting phase that allows ballot images to be viewed Provides the method to configure elections create ballot design add languages including audio export ballot election definitions and view ballot images e Election Reporting Module ERM a client application used for integrating election results acquisition consolidation and reporting A second ERM client can be configured
32. epartment of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Version Manufacturer Description Mouse Standard Mouse N A Monitor 1048x768 minimum N A resolution CD DVD reader 16x or faster N A USB Ports Minimum of 2 USB N A 2 0 ports Ethernet Port Available LAN N A connection Report Printer Standard Network N A Printer Rapidport 4 Digi Rapidport 4 USB Digi to 4 port Modem Network Security Device ASA 5505 Security Cisco Appliance Precinct Count Systems The precinct count scanner is the DS200 Digital Scan 200 DS200 e DS200 Hardware Revision 1 2 1 2 3 0 1 3 o 05200 Hardware v 1 2 3 0 is equivalent to HW 1 2 plus ECOs 1158 1160 1281 1346 1398 1400 Approved by BVSC on 7 31 13 e Firmware version 2 11 0 3 o Scanner Firmware v 3 0 0 0 o Power Management Firmware v 1 2 10 0 e Auxiliary equipment for 05200 o Plastic Ballot Box HW Revision 1 2 1 3 1 4 Plastic Ballot Bin HW v 1 4 is equivalent to HW 1 3 plus ECOs 1251 1266 1351 1388 Approved by BVSC on 7 25 13 and 7 31 13 o Steel Ballot Box With or without diverter o Plastic DS200 Tote Bin e USB Memory Device Media for DS200 o BDelkin USB 512MB 1GB 2GB 4GB 8GB e Compact Flash Memory Device Internal for 05200 o Delkin Compact Flash 1GB e Optional Modem Configuration 5 ECO Engineering Change Order process for initiating and controlling modifications to a sy
33. er of runs per test deck 15 Number of ballots per deck 320 Number of cards per ballot 1 Total number of ballots cast 192 000 Total number of vote targets 9 446 400 Findings The tested DS850s met the acceptance criteria for the central count scanner mass ballot count BVSC successfully scanned 192 000 ballots with 9 446 400 vote targets Page 19 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Two jams occurred on the DS850s however neither was attributed to the scanners One was due to a ballot with a folded corner when staff duplicated the ballot no jams occurred The other jam occurred due to a scanner part that was improperly adjusted after scheduled cleaning when the part was adjusted properly no jams occurred To mitigate risk of scanner jams ES amp S recommends that users follow procedures outlined in the DS850 Standard Operating Procedures manual to ensure proper cleaning and re assembly of the scanners Acceptance criteria is shown in the table below Table 7 Acceptance criteria for DS850 DS850 Mass Ballot Count Acceptance Criteria Expected Accepted Did the memory registers overflow No Y Did the public counters increment appropriately Yes Y Did the tabulated results agree with predetermined vote totals Yes Y Number of errors must not exceed 1 1 000 000
34. examined the DS200 precinct scanner and the ExpressVote for the ability to properly implement system time and date changes such as Daylight Saving Time and leap years Findings The DS200 properly performed an automatic reset for the spring spring forward change to Daylight Saving Time When the unit failed to automatically reset to the proper time fall back for the return to standard time the vendor was notified and responded with instructions for different testing conditions that accommodate the Linux code for time changes When re tested the DS200 satisfactorily reset automatically to standard time Both the DS200 and the ExpressVote properly handled leap year and non leap year date changes BVSC recommends the vendor update the documentation to clearly indicate how a user may test the DS200 s ability to automatically reset to standard time i e the fall back setting Page 28 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 Battery Life BVSC verified that the DS200 and the ExpressVote when disconnected from the electrical outlet functioned for the time period stated in the TDP documentation The documentation states that the battery pack is capable of powering the system for at least two hours 26 BVSC staff turned the power on the unit and disconnected the AC adapter The equipment was left running on battery power until the unit drai
35. fied several areas affecting usability and functionality that warrant enhanced documentation procedures training guidance or refinements to the system BVSC finds that with future enhancements the ExpressVote component could provide greater independence for persons with disabilities Therefore BVSC recommends provisional approval of the ExpressVote ballot marking device for use in any election conducted through May 1 2017 with the ExpressVote ballot marking device to be used solely by persons with disabilities The approval should be conditioned upon the vendor first making the necessary changes to its written documentation to reflect BVSC recommendations set forth in paragraph 1 under the subsection entitled Continuous Improvement and Recommendations for greater specificity clarity and guidance regarding use of the system The use of the ExpressVote ballot marking device if approved is also subject before use in any election to future revisions to Rule 1S 2 032 F A C to provide 33 ES amp S has satisfactorily completed EVS Release 4 5 0 0 Version 4 manual updates documentation requirements 34 ES amp S has satisfactorily completed manual updates documentation requirements as related to the ExpressVote ballot marking device Page 43 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 standards for the accessible audio visual onscreen display of the bal
36. fit within these limits Figure 14 While a paper ballot and onscreen displays use the main contest title or candidate name fields the ExpressVote summary card utilizes the alternate short name fields for contest and candidate Depending on the number of contests and names the ExpressVote may end up truncating these fields on the summary card 27 Counties may apply for an exception to the rule just as they may do so for paper ballots based on certified facts and circumstances justifying the deviation from rule requirements 28 This is a matter to be addressed in Rule 1S 2 032 Florida Administrative Code which will need to provide standards for this hybrid system Page 33 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Figure 14 Contest titles and candidates paper ballot vs ExpressVote summary card a m iu E BOLETA OFFICAL DE LAS BITLEN Vou OFISYEL d BALLOT ELECCIONES GENERALES ELEKSOW i MIAMI DADE COUNTY FLORIDA CONDADO DE MIAMI DADE M NOVEMBER 04 2014 FLORIDA 04 1 2014 m mmm m mmm m m m m m m m mm mm mm mmm EB p X PRECINCT 001 0 Q 0001 ut TO VOTE COMPLETELY FILL IN THE a cc v a Ape eS If you make a REPRESENTANTE ANTE EL CONGRESO DISTRICT COURT OF APPEAL a mistake cross through the mistake and fill in pip to e DIST
37. functionality to prevent inadvertent voting of the same card more than once by the same voter As a future enhancement the option to disable the AutoCast feature which is currently a setting at the unit level on each ExpressVote must be optionally overridden by a universal disable setting at the election definition level in ElectionWare Page 30 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 2 Contest selection on multiple ExpressVote cards Coding the ExpressVote for a multi card election poses a serious risk that a contest may be included more than once or not at all The contest selection process is a manual one in ElectionWare Figure 10 It is possible for the user to include a contest on more than one card or mistakenly exclude it from all cards Furthermore BVSC demonstrated that a voter could then vote the erroneous cards and therefore vote a contest more than once Figure 10 Contest selection for ExpressVote ballot styles Ballot Set Ballot Set Name Party ExpressVote Card 1 REP REPUBLICAN Ballot Set Type Election Day Absentee Early Ballot Style ID Contests Q9 By Precinct Combine Splits if the Same By Style Contests and Candidates All Except Local Local Only Exclude Nonpartisan Select Contests to Exclude COUNTY COURT JUDGE GROUP 4 COUNTY COURT JUDGE GROUP 9 Candidate Rotation COUNTY COUR
38. g modems in every polling place Page 25 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Findings Staff noted that the regional results reporting software prevents precinct results from being uploaded twice Staff did not observe any anomalies during the test Figure 8 Hotspots and Broadband Modems AT amp T Verizon Sprint 4 AC Sprint gt SPOTS BROADBAND MODEMS Page 26 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Figure 9 Connecting the Regional Reporting Site to the AT amp T Hotspot via Wi Fi technology REGIONAL RESULTS Although Florida law and FVSS do not require this test BVSC conducted a folded ballot test to simulate absentee ballot processing The objective was to observe the behavior of the precinct count and central count tabulators when folded ballots are scanned BVSC created a test deck for each of the four election types and four ballot lengths 11 inch PPP election 14 inch municipal election 17 inch primary election and 19 inch general election Different fold types were included in each test deck Z fold C fold fold through a vote target a fold through a write in and the maximum number of folds allowed per ballot length BVSC cast ballots into the DS200 and the DS850 and compared t
39. greater than 97 dB decibels weighted 13 Maximum volume minus minimum volume Must be greater than 20 dB 14 Must be between Minimum volume 12 dB and 97 dB 15 Must be greater than 97 dB decibels weighted 16 Maximum volume minus minimum volume Must be greater than 20 dB 17 Must be between Minimum volume 12 dB and 97 dB Page 21 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 requires that the force required to operate or activate the controls must be no greater than 5 pounds of force BVSC conducted these tests during an accessible voting session using a calibrated Dillon model GL digital force gauge Figure 5 BVSC set the force gauge to zero before each measurement taking three measurements for each button or touchscreen position Figure 5 ExpressVote undergoes testing with a force gauge AutoMARK Findings No measurement exceeded the maximum of 5 pounds of force BVSC found that the AutoMARK complied with section 101 56062 1 1 F S ExpressVote Findings No measurement exceeded the maximum of 5 pounds of force BVSC found that the ExpressVote complied with section 101 56062 1 I F S Page 22 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 During initial testing however BVSC discovered a problem with the audio tact
40. he results Findings The DS200 did not accept two 11 inch ballots one with a C fold and one with the maximum number of folds 2 The scanner displayed a ballot could not be read message This result is not considered an 24 Folds through a vote target are outside the ballot printing specification as outlined in the ES amp S Ballot Production Guide 25 The maximum number of folds per ballot length is determined by ES amp S TDP ballot printing specifications Lacking those BVSC uses the following 2 folds for 11 3 folds for 14 and 17 and 4 folds for 19 Page 27 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 error since the scanner gave the appropriate response The DS200 correctly processed all the other ballots The DS850 accepted all ballots except for the same two ballots that were rejected by the DS200 The reason for rejection by both scanners could be attributed to ballot fatigue a fold through a timing mark or some other common issue The DS850 also jammed on one 17 inch write in folded ballot The ballot was duplicated and the DS850 processed the duplicate correctly The DS850 processed all the other ballots correctly Scanner Sensitivity BVSC subjected all tabulator devices DS200 and DS850 to scanner sensitivity testing Florida law and FVSS have no requirements for this test so results are obtained for information purposes only Th
41. ile keypad and an ExpressVote software bug These findings are discussed in another section of this report see ExpressVote pg 30 Accessibility Clearance Requirements Voting booths or tabletop installations must meet minimum height and knee clearance dimensions as well as privacy requirements as set forth in section 101 56062 m F S The vendor s application includes an ExpressVote voting booth and an ExpressVote table with optional AutoCast box The ExpressVote may be used in conjunction with the ExpressVote voting booth which features a portable design as shown below Figure 6 ExpressVote voting booth Structural Foam Booth Snap on fabric privacy screen Srructural foam beckplate Small support ridge around Express Vote The ExpressVote may also be placed on an optional ExpressVote table with AutoCast box which allows the user to set up the ExpressVote to accept AutoCast ballots BVSC examined the AutoCast table without the privacy hood for compliance with applicable statutes The AutoCast table is shown in the figure below 13 AutoCast is a feature whereby the voter upon confirming and printing the vote summary card may have the card automatically placed into the secure AutoCast box attached to the table for later tabulation by election officials instead of having the card returned to the voter for casting into a precinct scanner Page 23 of 49 Florida Department of State Voting System Qualification
42. ing election activities election night and precinct level reporting post election and reporting activities BVSC used ballot test decks and pre determined results to compare to actual results Pre Election Activities Pre election activities included coding or verifying the coding of the election database for each of the four elections PPP municipal primary and general preparing the election media preparing the ballot test decks preparing and validating the expected results and preparing the voting equipment A universal primary contest UPC was included in the primary election definition Election Activities Election activities included opening polls casting ballots using test decks including hand marked ballots and ballots marked using the AutoMARK and the ExpressVote and closing polls Post Election and Reporting Activities BVSC examined test results against expected results Post election activities included generating reports This included uploading election results and verifying results in the ERM by election group absentee early voting Election Day and provisional voting BVSC uploaded results directly since modem testing was done in a separate test Partial and complete summary XML files and the 30 day precinct level XML files are produced by the XML File Utility which is ancillary to the voting system These are being examined in a testing activity separate from this certification effort Findings The
43. ions of the various bar codes that are used is shown below Figure 21 The ExpressPass pre prints the activation bar code the verification number corresponding to precinct or ballot style and the ballot header text If the user opts not to use ExpressPass the ExpressVote will print all as shown except the activation bar code and verification number after the voter has made his her selections and printed the vote summary card in the ExpressVote 31 ExpressVote Operator s Guide Firmware Version 1 2 Document Version 2 0 pg 30 Page 40 of 49 Florida Department of State Division of Elections Figure 21 Marked vote summary card 123456 MIAMI DADE CO FLORIDA PRESIDENTIAL GENERAL ELECTION NOVEMBER 6 2012 PRECEINCT 1 BALLOT STYLE 1 BEST AUTOMOBILE MANUFACTURER GENERAL MOTORS BESTVOCACARTIS oe FRANK SINATRA STEVIE RAY VAUGHAN BEST ICE CREAM FLAVOR NO SELECTION MADE PEACH m BEST NON CLASSICAL MUSIC ARTIST WHO IS NEITHER MALE NOR FEMALE IN A DUET NO SELECTION MAD Findings Voting System Qualification Test Report ES amp S EVS Release 4 5 0 0 Version 4 Activation Bar Code Verification Ballot Header Text Option 1 Printed by card printer at the same time as the activation bar code Shrink typeface size to fit the width without truncating Option 2 Not printed by the card printer but printed when the Vote S
44. ision of Elections ES amp S EVS Release 4 5 0 0 Version 4 Table 5 Acceptance criteria for DS200 DS200 Mass Ballot Count Acceptance Criteria Expected Accepted Did the memory registers overflow No v Did the public counters increment appropriately Yes Did the tabulated results agree with predetermined vote totals Yes v Number of errors must not exceed 1 in 1 000 000 vote targets An error is lt 1 1 vote defined as a target scan that produces a result other than the expected result targets Number of multiple feeds must not exceed 1 in 5 000 ballots A multiple feed lt 1 5K v occurs when the machine pulls multiple ballots and does not catch the error ballots Number of incorrect rejections of ballots must not exceed 3 amp v allots DS850 BVSC conducted a mass ballot count on two DS850 high speed central count scanners using the 2012 Escambia County Primary Election definition The minimum requirement was a ballot count of 192 000 ballots The mass ballot count test deck contained 40 sets of a single card 320 ballot test deck BVSC scanned the mass ballot count test deck 15 times to satisfy the 192 000 ballot count requirement Specific details follow Table 6 Mass Ballot Count details for DS850 Election definition used 2012 Escambia County Primary Ballot length 17 inches Number of scanner units used 2 Number of test deck sets 40 Numb
45. k L CIRCUIT RNEY 11TH JUDICIA 278 AT JEM KATHERINE FERNANDEZ RTH Gary Joh TOR DIS ives EP JOHN DANIEL COURIEL Virgil H Goode Jr James N Clymer PEAL 3 APPEAL 3 NAL DE Jill Stein JUSTI I undici DISTRICT COURT OF APPEAL 1 AP L fth RICT COURT OF APPEAL 2 jez ofthe General Information Observations ExpressVote Following are findings observed during various phases of the test process 1 Vote Summary Card list order display Prior to activating an ExpressVote voting session the ExpressVote vote summary cards equivalent to ballot styles are listed BVSC found that the sort order for this list is not consistent Sometimes Card 2 is listed first and sometimes Card 1 is at the top of the list Figure 15 No feature exists to systemically force a consistent card order Consequently this inconsistency increases the potential for selecting the wrong ballot style onscreen Page 34 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Figure 15 ExpressVote inconsistent card order While this inconsistency should not preclude approval of the ExpressVote additional procedural safeguards can mitigate the potential for human error This system inconsistency however should be eliminated before permitting an extension to the provisional approval of the ExpressVote if granted 2 Style Sheets process During ini
46. l Firmware SIB WinCE 5 00 20 Microsoft Operating System System Hardware 1 1 1 3 ES amp S Hardware revision for AutoMARK model A 200 00 1 1 1 3 ES amp S Hardware revision for AutoMARK model A 300 00 Compact Flash Memory SanDisk Toshiba Card Headphones AVID Components under Review The components under review include the following Upgraded ElectionWare election management system Upgraded precinct count tabulator DS200 Upgraded central count tabulator DS850 Ballot marking devices used by voters with disabilities o Upgraded AutoMARK o ExpressVote which is new to Florida Ancillary software Supported sizes include 512 KB 1 GB 2GB Stock headphones 5 The Technical Data Package TDP specifies the following headphones AVID Airline Products model number 1A6FVOGOCBK32ST ExpressVote Operator s Guide Firmware Version 1 2 Document Version 2 0 pg 24 The vendor supplied the following AVID educational headphones unmarked Page 15 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 o Toolbox o and ExpressPass thermal printer During the testing for certification ES amp S submitted numerous revisions to software and firmware to address issues discovered during testing BVSC examined all revisions One revision submitted by the vendor enhanced the ExpressVote s capability of handling multiple v
47. lot 2 Total number of ballots cast 10 800 Total number of vote targets 304 020 Findings The tested EVS 4 5 0 0 Version 4 DS200 firmware 2 11 0 3 met the acceptance criteria for the precinct scanner mass ballot count BVSC successfully scanned 10 800 ballots with 304 020 vote targets During testing of a previous system version however the DS200 presented an error in the poll closing process After staff performed the DS200 close polls function it presented a general system error along with two user options Turn off voting machine and Ignore and continue The user chose the latter as advised by the vendor and the DS200 then presented the This voting device is ready for poll to be opened screen with two options Don t open Turn off and Open Poll As advised by the vendor the user chose Open Poll The DS200 presented the Poll is open screen the public count was zeroed out and a zero tape printed The user was not given the option during this procedure to zero the public count nor did a results tape print before results were reset The vendor resolved the issue by modifying the certification application with an upgrade to the DS200 s firmware BVSC found no anomalies during the close polls process for Version 4 testing of the voting equipment Acceptance criteria is shown in the table below Page 18 of 49 Florida Department of State Voting System Qualification Test Report Div
48. lot and the printed vote summary card in accordance with section 101 5608 3 Florida Statutes Finally BVSC does not recommend an extension of this provisional approval if granted or recommend the approval of any future release of this ExpressVote ballot marking device unless the issues in paragraph 2 of the Continuous Improvements Recommendations section of this report are addressed or are no longer an issue 35 Any user who uses this system if certified is advised to take note of the recommendations herein to develop and implement procedures and provide explicit training to qualified personnel to minimize the potential risks or errors identified in this report particularly as follows a Add procedural controls and safeguards when selecting ExpressVote contests and verifying ballot styles b Train election officials explicitly to only use the ExpressPass and ExpressPass printer to minimize potential issues mentioned in paragraph 2 c amp d c Ensure proper cleaning and re assembly of the scanners Implement quality control procedures to reduce the scanning of worn ballots that could cause jams e Use care when entering data in the various name fields of ElectionWare as well as when verifying ballot styles due to the ExpressVote source fields and its truncating behavior f Take into consideration when calculating the quantities of ExpressPass printers to acquire and when developing their security and operational procedures that the
49. ned the battery and performed a shutdown operation BVSC reconnected the AC adapter and reviewed the audit logs to determine the length of time the machine remained in a usable state before complete drainage of the battery power occurred As many as ten votes were cast or marked in the case of the ExpressVote during the battery interval to demonstrate the unit s ability to operate on battery power The table below reflects the outcome of these tests Table 10 Battery life test results Battery Life Test Results Battery Life per TDP Actual Battery Life DS200 At least 2 hours 2 hours 26 minutes ExpressVote At least 2 hours 4 hours 32 minutes Findings The battery packs powered the DS200 and the ExpressVote for longer than the stated minimum battery life It is expected that in a real world scenario in which the unit would be utilized during the battery interval battery power would drain quicker relative to the number of ballots processed ExpressVote The ExpressVote is a voting device not currently certified in Florida The scope of the present application for approval is limited to use by any voter with a disability The ExpressVote offers two options once a voter has made his her selections and printed them on the human readable vote summary card either the voter may have the card returned to the voter who can then insert the card into a precinct or central count scanner for tabulation the voter can use an
50. ng processes thus resulting in extra votes for the contest Figure 12 Figure 12 Double contest on ExpressVote ballot and counted in election results ESCAMBIA PRIMARY Duplicate Race l 2 MARY MOCK ELECTION FLORII cate SUMMARY REPORT jlicate Race 8 26 2014 2811 ESCAMBIA PRIMARY cay tr De B Expressvote ExpressVote Ballot ii icm PREGDICT 827 SPLIT 1 PRECI minui HUM t wT Re tuetinSn 12 53 PADS LLLI REP PUBLIC DEFENDER ep Pom VOTES PERCENT 5 PERCENT SELECTION MADE COUNT ae BOARD OF COUNTY COMMISSIONERS DISTRICT ca GROUP t ERNANDEZ SELECTION MADE COUN PRECINCTS COUNTED OF 3 CIRCUIT JUDGE 11TH OITA CIRQUIT QUESTION NO SELECTION MADE D W LECTION MADE Cecilia Armenteros Chavez CIRCUIT JUOGE 11TH JUDICIAL CIRCUIT ATE Michael Bienstoc Jose L Joe Fernandez T SELECTION MADE CIRCUIT JUOGE 11TH CIRCUIT S Over Votes SELECTION MADE CIRCUIT JUDGE 11TH CIRCUIT Ate E NO SELECTION MADE COUNTY COURT JUDGE GROUP 1 COUNTY COURT JUOGE GROUP 4 COUNTY COURT JUDGE COUNTY COURT JUDGE COUNTY COURT JUDGE GROUP 11 COUNTY COURT JUDGE COUNTY COURT JUDGE COUNTY COURT JUDGE ES amp S documentation should be updated to advise users to implement explicit procedural controls when selecting contests and verifying ballot styles particularly when using the system in a multi ballot card election
51. on bar code and verification information printed on it via the ExpressPass printer With this information printed on the card the card can then be used to activate a voting session Page 38 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Figure 19 ExpressPass on a computer with ExpressPass thermal printer BVSC examined the ExpressPass functionality on a stand alone computer by importing the ballot styles using both the primary and general elections BVSC also analyzed the ExpressPass bar codes on the activation cards BVSC verified that the vote selections displayed on the ExpressVote s summary screen match the votes tabulated on the precinct and central count tabulators BVSC also verified that the ExpressVote reads the vote summary card independent of the ExpressVote unit used Page 39 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Figure 20 Voter activation card showing pre printed header to be inserted into ExpressVote BVSC COUNTYV FLORIDA 2811 ESCAMBIA PRIMARY 88 26 2814 PRECINCT 988 DEM PRECINCT 988 ExpressPass pre prints header information onto a blank vote summary card Figure 20 The pre printed header includes the activation bar code verification number and ballot header text A voted vote summary card with explanat
52. once by the same voter e Modifying the option to disable the AutoCast feature which is currently a setting at the unit level on each ExpressVote so that it can be optionally overridden by a universal disable setting at the election definition level in ElectionWare Conclusion The BVSC recommends certification of EVS Release 4 5 0 0 Version 4 voting system and provisional approval of the ExpressVote ballot marking device component as follows Certification of EVS Release 4 5 0 0 Version 4 voting system The BVSC recommends certification for EVS Release 4 5 0 0 Version 4 voting system except for the ExpressVote component which is discussed in the following section provided the vendor makes beforehand the necessary changes to its written documentation to reflect BVSC recommendations set forth in paragraph 1 under the subsection entitled Continuous Improvement amp Recommendations for greater specificity clarity and guidance regarding use of the system Provisional Approval of ExpressVote Ballot Marking Device The ES amp S ExpressVote represents new technology in a voting system specifically for use by Florida voters with disabilities For example the system introduces a vote summary card in lieu of a traditional ballot card upon which is imprinted only the voter s selections in all contests on a ballot all contest titles and a corresponding barcode to be read by the tabulator However in its current form BVSC has identi
53. onducted a recount in the primary election using the DS850 Both elections were single card elections Non recount races were suppressed in ElectionWare per the TDP instructions Results were verified against expected results Findings BVSC found that the voting system under test complied with applicable statutes and standards ElectionWare allows the user to report results from only the affected races Furthermore a recount can be conducted on more than one race at a time as demonstrated by processing both the countywide race and district wide race in one recount Modems BVSC examined the ability of the voting system to report and accumulate results from precinct scanners via modem communication Staff conducted a mock PPP election with four DS200 precinct scanners each using a different modem carrier or technology landline AT amp T Sprint and Verizon Findings During the setup of the modem test staff encountered difficulty getting the AT amp T modem to connect to the SFTP server The problem was eventually isolated to an Internet Access Point Name APN number in ElectionWare s DS200 transmission settings The APN number that worked in the vendor s home location Omaha NE did not work in the BVSC laboratory Tallahassee Once an APN number acceptable in this region of the country was entered in the ElectionWare settings the AT amp T modem capability 22 During subsequent testing of all modems staff observed no
54. ote summary cards for large multi ballot card elections The upgrade allows the voter s selections from all contests appearing on a ballot to be outputted on more than one vote summary card This is similar to a marksense ballot being comprised of several sheets Tests for this certification process were conducted with some elections that fit onto one card and some that fit onto multiple cards For clarity this report will identify tests as using single card or multi card elections where relevant Conduct of Tests Findings The test objective was to verify that this new voting system meets the applicable requirements of the Florida Voting Systems Standards FVSS Florida Statutes and Administrative Rules and the Help America Vote Act HAVA for usability and accessibility The FVSS qualification examination for this effort encompassed a physical and functional audit of the components under review BVSC conducted additional tests to verify compliance with standards for sound pressure levels and observe ballot sensitivity In addition BVSC conducted mass ballot count tests on precinct and central count tabulators as well as a mass marking test on the ExpressVote Systems Setup amp Configuration BVSC set up the ElectionWare voting system in two configurations as a client server configuration with modem and network capabilities and as a stand alone configuration Physical Audit BVSC conducted a physical audit to verify that the
55. own as ElectionWare a precinct scanner DS200 a precinct ballot marking device that can be used by voters with disabilities Auto MARK and a central count scanner 05850 Precinct results may be uploaded to the election management system manually or via wireless or landline modems This certification introduces a new device to Florida voters with disabilities The ExpressVote is a vote capture device with a verifiable paper record that is digitally scanned for tabulation This system combines paper based voting with touch screen technology The ExpressVote includes a mandatory vote summary screen that requires voters to confirm or revise their selections prior to printing the summary of ballot selections using the internal thermal printer Once printed ES amp S ballot scanners process the vote summary card The ExpressVote is designed to serve all voters but the EVS Release 4 5 0 0 Version 4 application request is for use only by voters with disabilities ES amp S has fully integrated the ExpressVote with the existing suite of ES amp S voting system products The Division of Elections Bureau of Voting Systems Certification BVSC conducted the certification testing in two phases Phase consisted of verifying the setup of the two configurations of the election management system restoration import of two elections general and primary elections creation of two elections municipal and Presidential Preference Primary elections a physical audi
56. partment of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 Hardware The hardware components of the voting system are listed and described below Election Management System Hardware The Election Management System hardware requirements are in the table below Table 2 Election Management System hardware requirements Product Version Manufacturer Description EMS Workstation PC workstation for Network Configuration Processor Dual Core N A RAM 2GB minimum 4GB N A recommended Hard Disk 150GB N A Keyboard Standard Keyboard N A Mouse Standard Mouse N A Monitor 1048x768 minimum N A resolution CD DVD reader 16x or faster N A USB Ports Minimum of 2 USB N A 2 0 ports Suggested network Ethernet Port Available LAN N A switch to connect to EMS connection Server Report Printer Standard Printer N A Gigabit Switch DGS 1008G D Link EMS Server Host system for Windows Server configurations Processor Quad Core RAM 4GB minimum Hard Disk 320GB Keyboard Standard Keyboard Mouse Standard Mouse Monitor 1048x768 minimum resolution CD DVD reader 16x or faster USB Ports Minimum of 2 USB 2 0 ports Ethernet Port Available LAN connection Report Printer Standard Printer Network Results Server Host system for network results server configuration Processor Quad Core N A RAM 4GB minimum N A Hard Disk 320GB N A Keyboard Standard Keyboard N A Page 10 of 49 Florida D
57. pport peripherals such as a sip and puff device or other assistive equipment Also the voter can use the AutoMARK to review a marked ballot and either to cast the ballot into an optical scan tabulator like the DS200 or if available to cast the ballot into an attached ballot box known as the AutoCast The DS200 is a voter interface device used to scan paper ballots It is a precinct tabulator that can also be used to scan absentee ballots This tabulator captures the voter s selections and digitally images the ballot The DS200 uses a USB drive for downloading the election definition provides an option to capture cast ballot images on the USB and provides the election results on the USB The results data can then be manually uploaded into ERM or the DS200 can transmit via secure wireless telecom network into ERM Figure 4 DS850 The DS850 optical tabulator is a high speed scanner for use to scan absentee ballots or to conduct machine recounts of contests races The DS850 uses digital cameras to image paper ballots capture voter selections on the image and evaluate the results It uses a USB drive for downloading the election definition captures the cast ballot images on the USB and provides the results on the USB The results data are uploaded into ERM either via the USB or via an Ethernet connection The DS850 also uses two COTS printers one for printing reports and the other for recording and printing an audit log The DS
58. sVote users to implement explicit procedural controls when selecting contests and verifying ballot styles in ElectionWare e To caution ExpressPass users to insert the vote summary card carefully into the ExpressPass printer so as to prevent misprints 2 Certification of future upgrades or enhancements to the EVS Release 4 5 0 0 Version 4 voting system or for extension of the provisional approval of the ExpressVote component if granted should be conditioned upon the vendor 32 Items 1 a 1 e have been satisfactorily addressed ES amp S submitted appropriately updated manuals documentation which were deemed by BVSC to be acceptable short term solutions to the listed items Page 42 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 a Eliminating the risk of errors in contest selection while programming an election b Programming a user warning and or confirmation dialog box in ElectionWare whenever text is used rather than data fields to generate the ballot styles This is to prevent discrepancies between voter instructions and vote for numbers c Improving the ballot card list order to be consistent so that the first card and subsequent cards in a multi card election appear in consistent order on the ExpressVote ballot style selection screen d Enhancing the ExpressVote multi card voting functionality to prevent voting of the same card more than
59. stem Page 11 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S EVS Release 4 5 0 0 Version 4 o Wireless Modem AT amp T Sprint Verizon o Analog Modem Multitech RJ 11 Precinct Voter Assistance Devices The voting system has two voter assistance devices the ExpressVote and the AutoMARK ExpressVote Voting System e Hardware Revision 1 0 o Input Output Board v 1 0 0 0 o Scanner Printer Engine SPE v 1 0 0 0 o Detachable Key Board DKB v 1 0 0 0 e Firmware version 1 2 3 0 e USB Memory Stick Media for ExpressVote o Delkin USB 512MB 1GB 2GB 4GB 8GB e Auxiliary equipment for the ExpressVote o Detachable key pad rocker paddle AVID headphones ExpressVote Booth o ExpressVote Table with optional AutoCast box AutoMARK Voter Assist Terminal VAT e Model A200 AutoMARK Hardware Revision 1 1 and 1 3 o Firmware version 1 8 5 0 o Printer Engine Board PEB v 1 70 o Scanner Interface Board SIB v 1 43 o WinCE v 5 00 20 o Ultra v 8 0 1 e Model A300 AutoMARK Hardware Revision 1 1 and 1 3 o Firmware version 1 8 5 0 o Printer Engine Board PEB v 1 70 o Scanner Interface Board SIB v 1 43 o WinCE v 5 00 20 o Ultra v 8 0 1 e Auxiliary equipment for AutoMARK o AutoMARK Voter Assistance Table o Plastic AutoCast Ballot Box HW Revision 1 0 Page 12 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ES amp S E
60. t and a functional audit Phase Il encompassed the conduct of mock elections and election cycle events such as loading the tabulators with the requisite media opening and closing of polls feeding ballots central count tabulation and election night and post election reporting BVSC performed tests to verify compliance with standards for sound pressure levels and contest recounts conducted mass ballot count testing for the central count scanner and the precinct scanner and conducted additional tests as necessary to observe the voting system s capabilities The ES amp S ExpressVote represents new technology in a voting system specifically for use by Florida voters with disabilities For example the system introduces a vote summary card in lieu of a traditional ballot card upon which is imprinted only the voter s selections in all contests on a ballot all contest titles and a corresponding barcode to be read by the tabulator BVSC finds that with future enhancements the system could provide greater independence for persons with disabilities However in its current form BVSC has identified several areas affecting usability and functionality that warrant enhanced documentation procedures training guidance or refinements to the system The BVSC recommends certification of EVS Release 4 5 0 0 Version 4 voting system and provisional approval of the ExpressVote ballot marking device component as follows 1 Certification for EVS Release 4 5
61. tial testing of the two card ballot in a primary election the DS200 correctly tabulated the first card s votes The votes on the second card however tabulated incorrectly as undervotes The vendor responded with instructions for the user to change the style sheets for the ExpressVote ballot styles Once the new instructions were followed the second page of the ExpressVote tabulated correctly BVSC recommends provisional approval conditioned upon the vendor modifying the user documentation to include the vendor s additional instructions regarding the user s use of the system s style sheets functionality 3 Manual selection of precinct ballot style To begin an accessible voting session on an ExpressVote programmed for early voting the election official must be present to manually select the precinct split if any and ballot style because there is no audio at that point in the session Figure 16 There is a potential risk that the election official may type in the wrong selections To avoid this problem ES amp S recommends that only the ExpressPass printer be used for multi card elections as the summary card will be imprinted with a bar code specific to the precinct split and ballot style Then when the summary card is inserted into the ExpressVote the precinct split ballot style are automatically selected and the voting session begins with the voter s language screen ExpressVote Operator s Guide Firmware Version 1 2 Document Version
62. ul Service RMS installation and removal of media ES amp S Event Log Service 1 5 4 0 ES amp S Provides custom event log UELS reports from the Windows Event Log VAT Previewer 1 8 5 0 ES amp S View on screen display of the AutoMARK in EMS ExpressVote Previewer 1 2 3 0 ES amp S View on screen display of the ExpressVote in EMS Page 7 of 49 Florida Department of State Division of Elections Product Version Election Reporting 8 10 0 0 Manager ERM ElectionWare Client ElectionWare All Components EMS Server Required for EMS server Software Firmware 4 5 0 0 Client 4 5 0 0 Standalone Voting System Qualification Test Report Manufacturer ES amp S ES amp S ES amp S EVS Release 4 5 0 0 Version 4 Description Election results reporting manager Election database ballot and election equipment configuration For Standalone only installs the Postgres database and ElectionWare client Windows Operating Server 2008 R2 Microsoft System Standard Edition Microsoft Offline 8 6 WSUS Updates Symantec Endpoint Small Business Symantec Protection Edition 2013 Cereberus FTP 6 0 7 1 Cereberus Optional RMCOBOL 12 06 Liant Removable Media 1 4 4 0 ES amp S Service RMS ES amp S Event Log Service 1 5 4 0 ES amp S UELS ElectionWare Server 4 5 0 0 ES amp S Operating System Offline operating system updates for security etc Anti Virus S
63. ummary Selections is printed Match typeface size Master Bar Code Vote Selection Bar Code BVSC found that the ExpressPass only prints one vote summary card at a time The ExpressPass printer quickly grabs the blank vote summary card and pulls it in to print the header If the user holds the vote summary card too tightly thus preventing the rollers in the printer from effectively grabbing the card the header may be improperly printed The user or operating manual should caution users to properly insert the vote summary card gently release it and allow the printer rollers to take control of the card Page 41 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 Toolbox The ES amp S Toolbox software application is a stand alone supplement to the ElectionWare voting system The Toolbox provides the following functions creating audio files through a text to speech interface and creating test decks using a 1 2 3 pattern BVSC conducted a cursory review of Toolbox s features Findings BVSC found no anomalies Source Code Review ES amp S submitted several source code revisions during certification testing after BVSC found several issues related to either security or operational issues BVSC checked the various source code versions and also examined them with the Klocwork static source code analysis tool BVSC determined that the latest version EVS 4
64. vote targets An error is lt 1 1 defined as a target scan that produces a result other than the expected result ios Number of multiple feeds must not exceed 1 in 5 000 ballots A multiple feed lt 1 5K occurs when the machine pulls multiple ballots and does not catch the error ballots Number of incorrect rejections of ballots must not exceed 3 s Ed v allots Accessibility Sound Pressure Level The sound pressure level test is conducted to verify conformance to section 101 56062 1 g i F S which describes the sound pressure level standards for a voting system s audio voting features BVSC conducted a sound pressure level test on the AutoMARK because the firmware has been upgraded since the previously certified version BVSC also conducted a sound pressure level test on the ExpressVote which is new to Florida and thus requires demonstration of conformance to the statute referenced above BVSC tested both machines using the AVID brand audio headsets supplied by the vendor BVSC used an ITU T P 50 test signal that was incorporated into an election definition The test signal replaced the initial sound file normally heard by a voter at the beginning of an accessible voting session The election definition played the test signal as a loop BVSC took all sound pressure level measurements after the elapse of a complete loop in order to capture instrument readings across the entire loop 7DS850 Operator Guide
65. voting system performed as indicated in the vendor s TDP and in accordance with FVSS Florida Statutes and Administrative Rules BVSC did however encounter some ExpressVote issues that the vendor should address for future extension of the provisional approval for the ExpressVote precinct marking device if granted or in a future system release Additional results of ExpressVote testing are discussed in another section of this report see ExpressVote pg 30 Precinct Tabulator Mass Ballot Counts BVSC conducted mass ballot counts on the DS200 precinct scanner and on the DS850 central count scanner Page 17 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 DS200 BVSC conducted a mass ballot count on one DS200 precinct scanner using the 2012 Miami Dade County General Election definition The minimum requirement was a ballot count of 9 900 ballots on a single scanner The test deck contained 360 two card ballots 720 cards total Staff ran the test deck through the DS200 fifteen times for a total of 10 800 ballots Specific details follow Table 4 Mass Ballot Count details for DS200 Election definition used 2012 Miami Dade County General Election Ballot length 17 inches Number of scanner units used 1 Number of test decks 1 Number of runs per test deck 15 Number of ballots per deck 360 Number of cards per bal
66. wing pre printed header to be inserted into ExpressVote 40 Figure 21 Marked vote summary neta nasa sagas assa nnn 41 Page 47 of 49 Florida Department of State Voting System Qualification Test Report Division of Elections ESGS EVS Release 4 5 0 0 Version 4 Tables Index Table 1 Election Management System software requirements sess enne 7 Table 2 Election Management System hardware requirements sess 10 Table 3 Voting system hardware components sss nennt nnn 14 Table 4 Mass Ballot Count details for 25200 nennen nennen nnn nnns 18 Table 5 Acceptance criteria for DS200 cccccccccccsssseceesssceeecsssceeecssseececseeesscsesececseasecsceeseceesesaeesesseaeees 19 Table 6 Mass Ballot Count details for 05850 19 Table 7 Acceptance criteria for 05850 vr e de dee eoe 20 Table 8 Sound pressure level test results AUtOMARK cccsccccesssseecessececesseeeeceseeeeeceeseeaecssseaaeeesseeaeens 21 Table 9 Sound pressure level test results ExpressVote 21 Table 10 Battery life test results eco icactei 29 Page 48 of 49 Florida Department of State KEN DETZNER Secretary of State
67. ystem Qualification Test Report Division of Elections 5 amp 5 EVS Release 4 5 0 0 Version 4 Figure 17 Contest coded for Vote for 2 DS850 tabulated Vote for 2 Over Votes Under Votes DS850 Results APPEAL 5 Vote For 1 YES NO Over Votes Under Votes Total COUNTY JUDGE GROUP 24 Countywide Vote For 2 Greer Elaine Wallace Andrea R Wolfson Thomas Carter Jon Adkins Randal Smith Write in Write in Over Votes Under Votes Total CC03 BOARD OF COUNTY COMMISSIONERS Cnty Comm 03 Vote For 1 The voter instruction on the paper ballot however read Vote for One whereas the voter instruction on the ExpressVote was displayed as Vote for no more than Two Figure 18 Figure 18 Paper ballot instruction Vote for One ExpressVote Vote for no more thanTwo PRECI 1 Paper ballot COUNTY JUDGE GROUP 24 JUEZ DEL CONDADO GRUPO 24 JIJ KONTE GWOUP 24 Countywide Vote for One Vote por uno Vote pou youn COUNTY JUDGE GROUP 24 Vote for no more than Two Greer Elaine Wallace 2 Card ExpressVote Election Greer Elaine Wallace 80 Health Ca Proposing Andrea R Wolfson 81 employer t lo purchas accept dire Thomas Carter 82 employers for lawful h Coverage c Jon Adkins services a permitted t laws or rule Randal Smith extent that directly for The problem was traced to a voter instruction manually entered in the ballot style Any extension of certification if granted should

Certification Test Report - Florida Department of State

Contents

Download Pdf Manuals

Related Search

Related Contents