Why Your Voting Machine Audit Is Probably Just Security Theater

TLDR: Most post-election audits are fundamentally broken because they rely on verifying electronic tallies rather than auditing the physical paper trail. Professor Philip Stark’s research demonstrates that true election integrity requires Risk-Limiting Audits (RLAs) that use statistical sampling to confirm outcomes with high confidence. Pentesters and researchers should stop treating electronic logs as the source of truth and start focusing on the physical, immutable evidence that actually determines the winner.

Election integrity is often framed as a software problem, but that is exactly where the industry goes wrong. We spend millions on hardening voting machines and securing e-poll books, yet we ignore the fact that these systems are often black boxes. If the software is compromised, the logs it generates are effectively useless. During his keynote at the DEF CON 2025 Voting Village, Professor Philip Stark laid out the mathematical reality of why most current audit procedures are little more than security theater.

The Detection Fallacy

Many jurisdictions claim to perform audits, but they are usually just checking if the machine’s internal math is consistent. They take a fixed percentage of precincts, re-tabulate the votes, and compare the result to the machine’s original output. This is a detection paradigm. It assumes that if the machine is working properly, the numbers will match.

The problem is that "working properly" is not a binary state. Machines fail, and they can be exploited. If an attacker modifies the code to flip a specific percentage of votes, a standard audit might never catch it because the machine is still "working" according to its malicious logic. You are essentially asking the fox to audit the security of the henhouse.

True security requires an affirmative evidence paradigm. Instead of asking if the machine is working, we should be asking if the paper ballots provide enough evidence to prove the reported winner actually won. This is the core of a Risk-Limiting Audit (RLA). An RLA does not care if the machine was hacked or if it suffered a hardware glitch. It only cares about the physical, hand-marked paper ballots.

Betting on the Outcome

Stark’s approach to RLAs is elegant because it strips away the complex statistical jargon and replaces it with a simple gambling analogy. Imagine you are betting on the outcome of an election. You start with a bankroll of one dollar. For every ballot you pull, you place a bet. If the ballot shows a vote for the reported winner, you win money. If it shows a vote for the loser, you lose money.

If you play this game repeatedly and your bankroll grows significantly, you have strong statistical evidence that the reported winner is correct. If you go broke, you have evidence that the reported winner might be wrong. This is not just a thought experiment. It is a rigorous application of Wald’s Sequential Probability Ratio Test (SPRT), a technique developed during World War II to make decisions with the minimum amount of data.

The beauty of this method is its efficiency. In a large jurisdiction like Orange County, California, you do not need to count three million ballots to verify the result. By using targeted sampling, you can confirm the outcome of hundreds of contests by looking at a tiny fraction of the total ballots cast. This makes it practical to audit every single contest in an election, rather than just the high-profile ones.

The Trustworthy Paper Trail

None of this works if the paper trail is garbage. If you cannot physically retrieve the specific ballot that corresponds to your sample, the audit fails. This is why the industry needs to move away from Direct-Recording Electronic (DRE) systems that do not produce a voter-verifiable paper record.

For a pentester, the takeaway is clear. If you are assessing a voting system, do not waste your time trying to find a buffer overflow in the proprietary tabulation software. Look at the physical process. How are ballots stored? Is there a chain of custody? Are the ballots uniquely identified in a way that preserves voter privacy but allows for retrieval? If the answer is no, the system is fundamentally insecure, regardless of how many patches you apply to the firmware.

Moving Beyond Security Theater

We see election officials "sprinkling magic RLA dust" on poorly run elections to claim they are secure. This is dangerous. An RLA is not a way to retroactively fix a broken process. If you do not have a trustworthy paper trail, you do not have an election; you have a performance.

Defenders need to stop focusing on the "robustness" of the software and start focusing on the auditability of the physical process. If you cannot prove the outcome using the paper, you cannot prove it at all. The next time you see a vendor touting their "secure" electronic voting platform, ask them how they handle an RLA. If they cannot explain the statistical sampling process or how they retrieve individual ballots, they are selling you a black box. Stop buying them.