Diving Into Our Straw Poll Data: Subjective Scales

Our June 30 10th CD Straw Poll gave voters the opportunity to use three unique voting methods—score, approval, and plurality voting—simultaneously on the same four-candidate congressional race. This gave us the opportunity to directly compare how voters interacted with each ballot.

In parts one and two of this series, we looked at bullet and anti-voting and min/max voting, respectively. In part three, we’re exploring subjective scales.

Subjective Scales

Score voting skeptics will often raise concerns that the “scale is subjective.” And it’s true, the score voting scale is subjective, but that may not be a meaningful complaint. In fact, a number of votes submitted at our straw poll implied that all voting may be subjective.

An example ballot showing a 1/5/1/1 score vote with a two-approval approval vote for the same candidates

For example, one of the straw poll ballots scored the candidates 1/5/1/1 on the score voting portion, but approved of two candidates on the approval voting portion. That voter explicitly gave all candidates except for their favorite the same low score, but nevertheless decided one of their less-preferred candidates still earned an approval vote. That voter apparently made the subjective decision that to approve one, and only one, of the three candidates they opposed.

An example ballot showing a 2/5/0/1 score vote with a bullet approval vote for the same candidates

An example ballot showing a 2/5/0/1 score vote with a two-approval approval vote for the same candidates

A separate set of four ballots scored the candidates 2/5/0/1 on the score voting portion. Three of those four voters approved of a single candidate—a bullet vote—on the approval voting portion. Remarkably, though, one of those voters also gave their second favorite candidate, the candidate that voter scored a 2, an approval vote.  These four voters all expressed the exact same levels of support for the candidates on the score ballot, but only one of them felt that two candidates deserved approvals.

Approval Thresholds

Even with our small sample size, we found voters approached setting their approval thresholds in a wide variety of ways. Of voters who cast bullet approval votes, the vast majority (87.0% or 20/23) effectively had a score of 5 as their approval threshold. Excluding bullet voters, the minimum score required to approve a candidate was fairly broad, with different voters approving candidates they scored anywhere from 1 to 5 out of a 0 to 5 range.

More than half of the voters who approved of more than one candidate (53.3% or 8/15) set their approval threshold to 3, just above the midpoint of the available ballot range. About a quarter of voters who approved of more than one candidate (26.7% or 4/15) approved of candidates they scored only a 2. Two more voters (13.3%) set their candidate approval threshold at a score of 4.

Lowest score for approved candidates by bullet and non-bullet voters in our CD10 Straw Poll

Scores cast by plurality choice in our CD10 Straw Poll

Mapping Score to Plurality

Comparing the plurality ballots to the score ballots shows a similar distribution, even though plurality ballots are restricted to a single mark. While all voters who gave a top score of 5 on the score voting portion of their ballot (71% or 27) also voted for that candidate on the plurality ballot, nine voters (24%) only scored their plurality choice a 4 and two voters (5%) only gave their plurality choice a score of 3.

This is one of the significant advantages of scoring-based ballots over ranking-based ballots: simply providing the ordering of the candidates, as ranked ballots do, obscures some of the voter’s intent. All of the voters provided a top score, by definition, but nearly a third of them did not believe their favorite candidate was worthy of the top grade. Score voting allows voters to easily express nuanced, albeit subjective, preferences.

Read parts one and two of our straw poll analysis and check back soon for part four: favorite betrayal.