The final March 4 Aggregate Average of the three groups polling the KS Caucus showed Trump (23.95%) leading Cruz (19.43%) with 28% undecided.
The chart below makes it very clear who the undecided voters broke for. (Click on the chart to expand it)
Because Trump, Rubio & Kasich’s numbers stayed relatively static from the final Aggregate Average of polling to the final March 5 result, it is easy to measure the accuracy & precision of each of the individual groups polling the race. The only meaningful changes from the March 4 Aggregate Average to 2:00 P.M. Saturday March 5 were Cruz and Undecideds.
The measure of the accuracy and precision of a poll is not how well it predicts a final result, but how accurately it reflects the reality on the day(s) the poll was taken. This is a difficult concept for some people to grasp because there are really are undecided voters. And there were a lot of undecided voters heading into the Kansas Caucus. Look at the chart above, add the 28% undecided voters to Cruz’s 19.43% and you get 47.43%–very close to Cruz’s final result.
The pollsters had five measures to get right on the day(s) their poll was taken: Trump, Cruz, Rubio, Kasich and Undecided Voters.
Docking’s poll was conducted 9 days before the Caucus. Freestate’s final poll was conducted the Sunday & Monday before Super Tuesday (before Carson even dropped out) and Trafalgar wrapped up the Thursday before the Caucus.
Freestate and Docking’s polls were conducted when Carson was still in the race. Trafalgar conducted its poll after Carson dropped out. To harmonize the data, Carson’s share in Docking and Freestate are added into the “Und/Other” category.
With five measurements (Trump, Cruz, Rubio, Kasich, Undecided) an individual measure of a candidate’s position within three points would be very accurate. Since all four polls had a margin of error about four we will say a measure within four is average and a total day-of-poll error less than twenty would be a good result. This creates three measures: The difference between the poll and the final results (Outcome Error) and Total Day-of-Poll Error and Average Day-of-Poll Error.
So, how accurate and precise were Trafalgar, Docking & Freestate’s polls on the days they were taken? (Click on table to expand)
Docking’s poll is pretty accurate considering it was taken 9 days before the Caucus. A lot of things happened in the race between Docking’s polling and March 5th. Docking accurately measured Trump and Rubio but errored widely on Kasich and Undecided voters. Docking had an Outcome Error of 48.14, a total Day-of-Poll Error of 21.84 with an average Day-of-Poll Error 5.46.
Trafalgar, which wrapped up polling about 36 hours before voting started, should have been the most precise and accurate. Trafalgar overstated Trump, was dead on with Rubio and very close on Kasich. Where Trafalgar failed was the number of undecided voters, which when combined with the overstatement of Trump led to an Outcome Error of 32.79, a total Day-of-Poll Error of 26.5 and an average Day-of-Poll Error of 6.62.
We at Freestate have repeatedly said that polling a Presidential Caucus is extremely difficult. It is one of the hardest things to get right in polling but on February 28-29 we got everything right nailing Trump, Rubio and Kasich. When Cruz’s numbers are added to Undecided they are within .71 of Cruz’s final total. Freestate had an Outcome Error of 33.77, Day-of-Poll total error of 2.23 with a Day-of-Poll average error of .55. Freestate has been very accurate in the past regularly nailing projection predictions within 2-3 percentage points but getting this many measures dead-on is 90% innovative and skillfully executed methodology and 10% chance.
Predicting the outcome of an election, especially when there is a large group of undecided voters, is a blend of polling and then running simulations similar to Monte Carlo Simulations based on the polling, base rate odds and some reasonable conjectures. As Nate Silver says, modern polling is more mathematical alchemy than old fashioned statistical sampling.
Freestate built its simulations off the aggregate averages of 8 different model weightings of Freestate’s February 15-16 & Feb 28-29 polls (Jeb was still in the race for the Feb 15-16 poll!), Docking and, in the final projection, Trafalgar. The more accurate the initial polls, the more accurate the resulting simulations and projections.
Before Trafalgar was added to the simulations, Freestate’s projections were trending the correct direction showing a Cruz win (scroll to the end of the report).
But the simulations could not sustain the added errors of Trafalgar.
In the other March 5 GOP Primaries the undecided voters broke for Cruz. As the march to the GOP nomination continues, keep an eye on the undecided voters and if Kasich or Rubio drop out, Cruz could begin racking up the wins.