Risk Limiting Audits

California Code of Regulations

Title 2. Administration

Division 7. Secretary of State

Chapter 2. Risk Limiting Audits

20110. General Provisions.

1. The purpose of this chapter is to establish guidelines and procedures for an elections official to conduct a risk-limiting audit in accordance with the requirements of Division 15, Chapter 4, Article 5.5 of the Elections Code.
2. Pursuant to Elections Code section 15367(a)(1), commencing with the statewide general election held on November 3, 2020, the elections official conducting an election may conduct a risk-limiting audit during the official canvass of any election.
3. A participating county may exclude any contest that has been subject to a risk-limiting audit from the one-percent manual tally required by Section 15360.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 15150, 15360 and 15367, Elections Code.

20111. Definitions.

As used in this Chapter, the following terms have the following meanings:

1. “Ballot” shall have the meaning set forth in Elections Code section 15366(a). A validly cast ballot is any ballot that is eligible to be counted in the canvass of an election, as specified in Division 15, Chapters 2 through 4 of the Elections Code.
2. “Ballot card” means a card upon which is printed, or identified by reference to the ballot, the names of candidates for nomination or election to one or more offices or the ballot titles of one or more measures. A ballot may be comprised of multiple ballot cards.
3. “Ballot-level comparison audit” shall have the meaning set forth in Elections Code section 15366(b). In performing this type of risk-limiting audit, the elections official shall export a cast vote record from the voting system for every cast ballot card and cast voter verified paper audit trail containing the contest(s) under audit. The cast number of cast vote records must be exported in a way that enables the elections official to determine whether the number of cast vote records matches the number of ballot cards physically accounted for in the ballot manifest and to find the particular physical ballot card or voter-verified paper audit trail associated with each cast vote record.
4. “Ballot manifest” means a detailed description of how the ballot cards are stored and organized, listing at minimum the unique physical location of each and every ballot card cast in the election in such a way that individual ballot cards or batches of ballot cards can be found, retrieved, and examined manually.
5. “Ballot polling audit” shall have the meaning set forth in Elections Code section 15366(c). In examining voter markings, the audit board shall perform a hand-to-eye, human interpretation of voter markings but shall not compare those interpretations to the corresponding cast vote records.
6. “Batch comparison audit” means a type of risk-limiting audit in which the audit board examines voter markings on ballot cards, in randomly selected batches of ballot cards. This type of risk-limiting audit involves the following steps:
   1. The elections official divides the ballot cards cast in the contests under audit into physically identifiable batches, such as the ballot cards cast in different precincts or vote centers.
   2. The elections official exports vote subtotals from the voting system for each such physically identifiable batch of cast ballot cards.
   3. The elections official verifies that the number of ballot cards according to these subtotals does not exceed the number of ballot cards physically accounted for in the ballot manifest.
   4. The elections official compares some or all of those batch-level subtotals to subtotals derived by hand-to-eye, human interpretation of voter markings from the corresponding ballot cards marked by the voter or the voter verified paper audit trail, as defined by Elections Code section 19271.
7. “Cast vote record” shall have the meaning set forth in Elections Code section 15366(d). The cast vote record shall be generated by the voting system.
8. “Elections official” shall have the meaning set forth in Elections Code section 320.
9. “Electoral outcome” shall have the meaning set forth in Elections Code section 15366(e).
10. “Hybrid audit” means a type of risk-limiting audit which involves, at a minimum, the following steps:
    1. The elections official divides the ballot cards cast in the contests under audit into physically identifiable batches, such as the ballot cards cast in different precincts or vote centers or ballot cards processed by different voting equipment.
    2. The elections official exports vote subtotals or cast vote records from the voting system for some of those batches of cast ballot cards from paragraph (1) and exports, at a minimum, aggregate results for the remaining batches of ballot cards.
    3. The elections official verifies that the number of ballot cards according to the subtotals in paragraph (2) does not exceed the number of ballot cards physically accounted for in the ballot manifest.
    4. The audit board compares some or all of the vote subtotals or cast vote records exported in paragraph (2) to a tabulation or interpretation derived by hand-to-eye, human interpretation of voter markings from the corresponding ballot cards marked by the voter, as defined by Elections Code section 19271.
    5. The audit board examines some or all of the ballot cards in the remaining batches from paragraph (2) and performs a hand-to-eye, human interpretation of voter markings from the corresponding ballot cards marked by the voter, as defined by Section 19271.
11. “Public notice” means the release of information to the public through two or more of the following readily available communication channels: a website update, a social media post, an email list mailing, a press release, and a notice posted at an office open to the public.
    1. Elections officials are encouraged to maintain an email mailing list of persons who would like to be notified about risk-limiting audits. If an elections official maintains such a list, public notice for the purposes of this Chapter must include sending an email to all persons on this list.
12. “Random seed” means a randomly generated number consisting of at least 20 digits that is used to generate a random number sequence to select ballot cards or batches of ballot cards for audit.
13. “Risk-limiting audit” or “RLA” shall have the meaning set forth in Elections Code section 15366(f).
14. “Risk-limiting audit software tool” or “RLA software tool” means software that can perform any of the audit types described in Section 20112. Pursuant to Elections Code section 15367(b)(2)(E), the algorithms and source code of the RLA software tool shall be disclosed to the public. The Secretary of State shall effect this disclosure by posting the source code, or a link to the source code hosted on another publicly available website, on its website.
15. “Voting system” shall have the meaning set forth in Elections Code section 362.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 320, 362, 15302, 15366, and 15367, Elections Code.

20112. Audit Types.

The elections official shall use an RLA software tool provided by the Secretary of State to perform one of the following:

1. A ballot-level comparison audit, with a five percent risk limit.
2. A ballot polling audit, with a five percent risk limit.
3. A batch comparison audit, with a five percent risk limit.
4. A hybrid audit, with a five percent risk limit.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20113. Audit Initiation.

1. If the audit commences after all validly cast ballot(s) within the jurisdiction have been tabulated, the audit shall be conducted in a single phase as described in section 20121.
2. If the audit commences before all validly cast ballot(s) within the jurisdiction have been tabulated, it shall be conducted in two phases as described in section 20122.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20114. Contests to be Audited.

Participating counties shall conduct a risk-limiting audit on one or more contests fully contained within the county’s borders. Participating counties may conduct opportunistic reviews of contests not subject to audit, including cross-jurisdictional contests, from the same sample of ballot cards by entering voter selections from those contests into the RLA software tool. The results of opportunistic reviews shall not be considered confirmation to any risk limit of the outcomes of these contests.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 15150 and 15367, Elections Code.

20115. Audit Board Selection.

1. The elections official shall appoint an audit board(s) to perform the duties specified in section 20123 that meets the following criteria:
   1. Audit boards shall consist of no fewer than three members. At all times, at least one member shall serve as an observer of the audit and may not make determinations of voter choices. At least two members, excluding the observer, shall make determinations of voter choices for each ballot card examined by the audit board. Audit board members may rotate roles.
   2. Audit board members may be comprised of election office full-time or temporary staff, volunteers or a combination of staff and volunteers. When feasible, audit board members involved in adjudicating voter intent during the tabulation shall not be involved in ascertaining voter intent during the audit.
   3. Prior to conducting the audit, each member of an audit board shall complete and sign a declaration of intent to faithfully discharge audit board duties. Each audit board member shall do so in the presence of the elections official on a form provided by the elections official. The elections official shall countersign the declaration. The declaration shall be in the following form:
   4. No fewer than two members of the audit board will be in possession of ballot cards at any time, including during the ballot card retrieval process.
2. If the elections official appoints multiple audit boards, only one audit board shall evaluate each ballot card.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20116. Public Education on Risk-Limiting Audits.

1. Prior to conducting an RLA, the elections official shall provide public notice that includes descriptions of how the process will be conducted, the difference between 1% manual tally and an RLA, and the methods for ensuring ballot security. The elections official shall also provide information regarding the process for selection of their audit boards and procedures used to ascertain voter intent manually.
2. The Secretary of State shall make the same materials described in (a) above available on its website.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20117. Ballot Manifest and Ballot Handling.

1. The elections official shall maintain an accurate ballot manifest, created without reliance on the voting system. The ballot manifest shall uniquely identify the storage container in which each validly cast ballot card is stored after tabulation.
2. The ballot manifest shall be in the format required by the RLA software tool in the California Post-Election Risk-Limiting Audit Ballot Manifest Format document dated October 15, 2019, which is hereby incorporated by reference, and which the Secretary of State shall post on its website.
3. If the audit commences after all validly cast ballot(s) have been tabulated, the elections official shall create only one ballot manifest.
4. If the audit commences before all validly cast ballot(s) have been tabulated, the elections official shall create two ballot manifests, one before each phase of the audit: an initial ballot manifest that includes all ballot cards that have been tabulated before the first phase of the audit starts, followed by the final ballot manifest including all tabulated ballot cards.
5. The content of the initial ballot manifest shall not be changed in the final manifest; accordingly, no ballot cards shall be added to the containers included in the initial manifest. Instead, the tabulated ballot cards that are not included in the initial ballot manifest shall be stored in new containers, and the final ballot manifest shall include all the rows in the initial ballot manifest plus a row for each new container. Example: 1,000,000 ballot cards were cast in the county, of which 900,000 had been tabulated when the elections official decided to start the first phase of a two-phase audit. The initial ballot manifest includes those 900,000 ballot cards. Ultimately, 97,000 of the remaining 100,000 ballot cards are determined to be validly cast. These ballot cards are tabulated and are stored in new containers. The final ballot manifest consists of the initial ballot manifest plus additional rows that describe the new containers in which these 97,000 ballot cards are stored, for a total count of 997,000 ballot cards in the ballot manifest for the second audit phase.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20118. Chain of Custody.

1. The elections official shall maintain and document uninterrupted chain of custody for each ballot card and each ballot storage container. Chain of custody logs should, at a minimum, include an identifying number of each ballot storage container and the number of a tamper-evident seal affixed to each ballot storage container. Chain of custody logs shall be available for public inspection after the canvass of the election pursuant to Division 15, Chapters 2 through 4, of the Elections Code is complete.
2. The elections official shall secure and maintain in sealed ballot containers all tabulated ballot cards.
3. The elections official shall establish written procedures to ensure the security, confidentiality, and integrity of any ballot cards, cast vote records, or any other data collected, stored, or otherwise used pursuant to this section. These procedures shall be published on its website at least five days in advance of the audit.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20119. Data Publication Prior to Audit.

1. When the elections official submits data to the RLA software tool before an audit phase begins they shall also publish the same data on their website before continuing with the audit. The elections official shall also publish any changes made to such data.
2. Subject to Elections Code section 2194(a), cast vote record data shall not be posted to the elections official’s website, but shall be made available to the public at the location where the audit is being conducted. The cast vote records made available to the public must include all contests selected for audit, and may also include cast vote records for any other contest in the election not subject to the audit.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 2194 and 15367, Elections Code.

20120. Random Seed.

1. The random seed shall be generated in order by sequential rolls of one or more fair 10-sided dice.
2. The random seed shall be generated in a public meeting as follows:
   1. The elections official shall give at least five days public notice of the public meeting before generating the first random seed and public notice at least one hour before generating a second random seed, should a second random seed be needed when conducting a two-phase audit.
   2. The elections official shall randomly select members of the public who attend the meeting to take turns rolling a die. At the election official’s discretion, the elections official may designate one or more staff members to take turns rolling a die.
   3. In the event that no members of the public attend the meeting, the elections official may designate someone or themselves to roll the die.
   4. Members of the public attending the public meeting shall be permitted to video record or photograph the random seed generation.
3. After the random seed is generated, the elections official shall provide public notice of the random seed.
4. The random seed shall be entered into the RLA software tool as provided in sections 20121(d) and 20122(e).

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20121. Audit Procedures for Single-Phase Audit.

1. The elections official conducting an RLA after all ballot cards cast within the jurisdiction have been tabulated and reported shall enter the following information into the RLA software tool:
   1. The ballot manifest for all ballot card(s);
   2. The results for all ballot card(s) tabulated;
   3. For ballot-level comparison audits, the cast vote records for all ballot card(s) tabulated;
   4. For batch comparison audits, vote totals from the voting system for each contest being audited for each batch; and
   5. For hybrid audits, all vote subtotals or cast vote records that were exported pursuant to paragraph (2) of subdivision (j) of Section 20111.
2. If the RLA software tool identifies any inconsistencies in the information entered under subdivision (a), the elections official shall, if possible, resolve the inconsistencies before the audit proceeds and note the inconsistencies and their cause(s) in the final audit report in section 20125.
3. After the data have been submitted under subdivisions (a) and (b), the elections official shall generate a random seed pursuant to section 20120.
4. The elections official shall enter the random seed into the RLA software tool. The RLA software tool will generate a list of particular ballot card(s) or batches of ballot card(s) from the ballot manifest to examine manually.
5. The elections official may at any point decide to conduct a full manual tally of any contest(s) whose outcomes have not yet been confirmed by the RLA. In the event that the elections official conducts a full manual tally, the RLA of that contest shall be suspended. Such a manual tally shall follow the procedure specified in Elections Code section 15290.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 15290 and 15367, Elections Code.

20122. Audit Procedures for Two-Phase Audit.

1. An elections official conducting a two-phase audit under subdivision (d) of section 20117 shall do so in accordance with this section.
2. In the first phase of the audit, the elections official shall enter the following information into the RLA software tool:
   1. The initial ballot manifest for all ballot card(s) tabulated as described in subdivision (d) of section 20117;
   2. The results for all ballot card(s) tabulated that are included in the initial ballot manifest;
   3. For ballot-level comparison audits, the cast vote records for all ballot card(s) tabulated that are included in the initial ballot manifest;
   4. For batch comparison audits, vote totals from each contest being audited for each batch;
   5. For hybrid audits, all vote subtotals or cast vote records that were exported pursuant to paragraph (2) of subdivision (j) of Section 20111; and
   6. The maximum number of ballot cards that remain to be tabulated.
3. If the RLA software tool identifies any inconsistencies in the information entered under subdivision (b), the elections official shall, if possible, resolve the inconsistencies before the audit proceeds and report the inconsistencies and their cause(s) in the final audit report in section 20125.
4. After the data have been submitted under subdivisions (b) and (c), the elections official shall generate the first random seed pursuant to section 20120.
5. The elections official shall enter the first random seed into the RLA software tool. The RLA software tool will randomly generate a list of particular ballot card(s) or batches of ballot card(s) from the ballot manifest to examine manually.
6. In the second phase of the audit, the elections official shall enter the following information into the RLA software tool:
   1. The final ballot manifest for all ballot card(s) tabulated as described in subdivision (d) of section 20117;
   2. The results for all ballot card(s) tabulated that are included in the final ballot manifest;
   3. For ballot level comparison audits, the cast vote records for all ballot card(s) tabulated;
   4. For batch comparison audits, vote totals from each contest being audited for each batch; and
   5. For hybrid audits, all vote subtotals or cast vote records that were exported pursuant to paragraph (2) of subdivision (j) of Section 20111.
7. If the RLA software tool identifies any inconsistencies in the information entered under subdivision (f), the elections official shall, if possible, resolve the inconsistencies before the audit proceeds and report the inconsistencies and their cause(s) in the final audit report in section 20125.
8. After the data have been submitted under subdivisions (f) and (g), the elections official shall generate the second random seed pursuant to section 20120.
9. The elections official shall enter the second random seed into the RLA software tool. The RLA software tool will identify whether further auditing is required to meet the risk limit. If further auditing is required, the RLA software tool will randomly select additional particular ballot card(s) from the final ballot manifest to be examined manually.
10. The elections official may at any point decide to conduct a full manual tally of any contest(s) whose outcomes have not yet been confirmed by the RLA. In the event that the elections official conducts a full manual tally, the RLA of that contest shall be suspended. Such a manual tally shall follow the procedure specified in Elections Code section 15290.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 15290 and 15367, Elections Code.

20123. Ballot Retrieval and Manual Examination.

1. The audit board shall locate and retrieve, or observe the location of and retrieval by elections official or election staff of, each randomly selected ballot card from the appropriate storage container. The audit board shall verify that the seals on the appropriate storage containers are those recorded on the applicable chain of custody log.
2. The audit board shall examine each randomly selected ballot card. If the selected ballot card was duplicated prior to tabulation, the audit board shall retrieve the original ballot card and report how the original (rather than the duplicate) was marked.
3. The audit board shall interpret voter markings on ballot cards selected for audit in accordance with Elections Code section 15154. If the audit board members cannot unanimously agree on the voter's intent, they shall indicate their disagreement in the final report in section 20125. They shall then notify the elections official of the disagreement, who shall arbitrate the issue. The elections official may:
   1. Instruct the audit board members to replace the ballot card with another ballot card, selected at random, that contains the same contests as the previously selected ballot card;
   2. Instruct the audit board members to submit to the RLA software tool that no consensus can be reached on the voter choices; or
   3. Make a final determination of the voter markings and instruct the audit board to submit the voter markings or choices in all audited contests to the RLA software tool.
4. The audit board shall record the voters' choices in every contest on every ballot card selected for audit. Those choices shall be entered into the RLA tool.
5. The audit shall continue until the risk limit is met for every contest being audited pursuant to section 20114. If the RLA software tool specifies that further auditing is required to meet the risk limit, it will randomly select additional ballot card(s) from the ballot manifest to be examined manually. The elections official shall retrieve those additional ballot card(s) as described in (a), examine those additional ballot card(s) as described in (b) and (c), and record the voters' choices as described in (d).
6. If the elections official chooses to perform a full manual tally of one or more contests, the audits of those contests will be deemed to have met the risk limit and the results of the full manual tally shall be the official results for those contest(s).
7. The RLA shall be completed no later than the canvass deadline, as specified by Section 15372 of the Elections Code.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 15154, 15360, and 15367, Elections Code.

20124. Public Observation and Verification of Audit.

1. The elections official shall ensure that the audit process is observable and verifiable by the public. The elections official shall:
   1. Provide at least five days public notice prior to the first phase of the RLA. This notice shall include the time and place of the random seed generation described in section 20120 and the date the ballot cards will be retrieved and manually examined.
   2. Describe in writing the process of manually examining ballot cards and the selection of the ballot cards to be used in the audit.
   3. Provide observers with an oral and/or written explanation of the RLA process, a written code of conduct for observation, and any documentation they will need for informed and effective observation.
      1. The code of conduct for observation shall be determined by the elections official. The code of conduct for observation shall explain the rights and responsibilities of observers.
      2. Documentation needed for informed and effective observation shall include but not be limited to any data the audit relies upon, including:
         1. The ballot manifest;
         2. For ballot-level comparison audits, the cast vote records;
         3. For batch comparison audits, vote totals from each contest being audited for each batch; and
         4. For hybrid audits, all vote subtotals or cast vote records that were exported pursuant to paragraph (2) of subdivision (j) of Section 20111.
   4. Disclose the methods used to select samples and to calculate the risk.
   5. Provide the public the opportunity to observe ballot cards being retrieved and examined during the audit.
   6. Provide the public the opportunity to observe the voters’ marks on every audited ballot card during the audit.
2. Observers may ask questions of the audit board provided they do not interfere with the conduct of the audit. The elections official, at their discretion, may designate an alternate person to receive and respond to observer questions in lieu of the audit board, and this person shall confer with the audit board to obtain answers to questions to which this person does not know the answers.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20125. Certification of Contest Results and Reporting of Audit Results.

1. The elections official conducting an RLA shall report the results of the audit in the certification of the official canvass of the vote specified by Section 15372 of the Elections Code. The report shall contain the following:
   1. The type of RLA conducted (e.g. ballot comparison, ballot polling, batch comparison, or hybrid);
   2. The date and time the RLA commenced and finished, and the number of rounds of sampling;
   3. A list of contests audited, with final reported results for those contests and the final measured risk for each contest;
   4. A link to the website where the ballot manifest can be found;
   5. The random seed(s) the elections official generated for use with the RLA software tool, and when and how the random seed(s) was generated;
   6. Any inconsistencies between the number of ballot cards and the number of cast vote records, or inconsistencies in the data reported by the RLA software tool, and—when possible—the reason(s) for those inconsistencies.
   7. The number of ballot card(s) examined for each contest under audit, noting whether a full manual tally was conducted. If the audit was conducted in stages or continued beyond the first sample, a description of how the sample was drawn and the number of ballot card(s) selected at each stage.
   8. Notes regarding any ballot card(s) for which the audit board could not determine the voter’s intent, as described in subdivision (c) of section 20123;
   9. A link to the website where the list of the particular ballot card(s) examined can be found;
   10. For ballot-level comparison audits:
       1. The number of discrepancies between ballot card(s) and the cast vote record, and a description of each discrepancy; and
       2. A link to the website where the cast vote record file only for the ballot card(s) examined in the audit can be found;
   11. For batch comparison audits, discrepancies between batches and batch totals, and a description of each discrepancy;
   12. For opportunistic reviews of contests not subject to audit, as described in Section 20114, a list of contests reviewed and the result of the reviews.
   13. The person-hours required to prepare for the audit, and number of people involved;
   14. The person-hours required to conduct the audit, and number of people involved;
   15. The number of members of the public who observed the audit;
   16. Notes on anything unusual or problematic, or that would be useful to improve the process, or that might be of value to the Secretary of State or other elections officials; and
   17. An attestation from the elections official that the RLA was conducted in accordance with Title 2, Division 7, Chapter 2 of the California Code of Regulations.
2. The certification of the official canvass of the vote shall be conducted in accordance with Section 15372 of the Elections Code.
3. The Secretary of State shall publish any report under this section received from an elections official on its website no later than five days after the date it publishes the certification of the official canvass of the vote in accordance with Section 15372 of the Elections Code. This subdivision shall not prohibit the elections official from publishing the same report on its website.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15367, Elections Code.

20126. Redaction of Personally Identifiable Voter Choices.

No later than the third business day following the expiration of the deadline to request a recount under Elections Code sections 15620 or 15621, or the completion of any recount, whichever is later, an elections official who conducted an RLA shall review its cast vote records file that was published and redact from the public record, but not its own files, the voter choices corresponding to any ballot susceptible to being personally identified with an individual voter.

Note: Authority cited: Section 12172.5 Government Code; Section 15367, Elections Code. Reference: Sections 15367, 15620, and 15621 Elections Code.