As America’s pollsters and poll aggregators conduct their post-mortems on the 2024 presidential election, some are already saying the polls got it right, noting close results in each of the seven presidential contests.
But there’s one key point that can’t be missed: The polls have again underscored the depth of support for President-elect Donald Trump, despite many changes pollsters have made since their 2020 and 2016 releases.
To be sure, this year’s polls appear to have missed fewer targets overall than four years ago, and the results in swing states were close enough to be within the margin of error for a significant number of polls. NBC News Decision Desk analysis.
But the loss of Trump supporters in the public polls meant that early polling averages didn’t show Trump sweeping swing states, so the result came as a surprise to some, even though we probably shouldn’t have. he was surprised.
NBC News compared Trump’s support in “likely voter” polls conducted in October and November with his vote share in state and national elections. The pattern is similar to what we’ve seen in pre-election polls in the previous two presidential elections: The median poll lowered Trump’s support almost everywhere, and in seven swing states, the miss was consistently between 2 and 3 percentage points.
Pollsters can take some comfort from the fact that voting averages in state-level presidential elections are slightly better than in 2020 — perhaps indicating that voting adjustments are helping to limit overall polling error. In an analysis of all polls released to the public in the last two weeks of the 2020 election, these pre-election polls in the last two weeks showed Trump’s support an average of 3.3 percentage points lower than the final results. In 2024, polls in the last two weeks showed his support down by an average of 2.4 points.
The chart below shows the difference between polls and results nationally, as well as the number of polls conducted in October and November.
This underestimation of Trump’s support spanned polls across the political spectrum, from solidly Democratic states (New York, down 4.6 points) to states that are “purple” in Democratic dreams (Texas, down 4.4 points). solidly Republican states (Wyoming, 5.8 points too low), swing states (Nevada, 2.9 percentage points too low). That’s despite attempts by pollsters to explain the difficulty of Trump supporters answering their polls.
Whether the polls offer the right “vibe” about the race depends on where you look. In the critical swing states – Arizona, Georgia, North Carolina, Michigan, Nevada, Pennsylvania and Wisconsin – many polls suggested either a close race or a 1-point race. Given the margins of sampling error and other sources of error in voting, these results left plenty of room for the final results to swing one way or the other. But the closely divided polls surprised some when Trump ended up sweeping all seven states.
Most polls correctly showed Trump winning Arizona, Georgia and North Carolina, but did not show him leading in Michigan, Nevada, Pennsylvania or Wisconsin. And even when the swing state polls correctly identified Trump as the frontrunner, a significant number of polls overestimated his support by more than the margin of error.
For example, Trump had the lead in 85% of Arizona polls, but 36% of Arizona polls showed his lead below the margin of error.
Reasons for separation
Well, what happened? There are likely to be two culprits.
First, perhaps the polls — once again — did not capture enough new voters or voters who switched from supporting Biden in 2020 to supporting Trump in 2024. Talk to survey takers as we see them in 2020.
Public opinion is difficult, if not impossible, to measure if the people who respond to surveys differ in their opinions from those who do not, especially in ways or to an extent that the pollsters did not expect. Voters who feel disrespected or misunderstood when they share their opinions with reporters or pollsters may choose to avoid them altogether.
Second, pollsters trying to predict what the 2024 electorate will look like may have simply made incorrect assumptions, leading to voting error like the one we’re seeing now. This illustrates a challenge inherent in early voting: the need to adjust pollsters’ data to match what they want think the voter will be without knowing whether those amendments are correct or not.
But if the 2024 electorate changed in a way that didn’t account for the pollsters’ assumptions—the corrections would be inadequate.
For example, in 2024, many pollsters began weighting respondents’ self-reported past voting (that is, whether people said they voted for Trump or Biden in the last election) to match their 2020 results. This is the statistical correction the pollster used this cycle for the earlier undercount of Trump voters. When taking polls to compare the popular vote four years ago, pollsters assumed that Biden voters in 2020 and Trump voters would vote at the same rate in 2024 — but only if Trump voters were slightly more likely to vote and Biden voters slightly more likely to vote. if less. poll, it could easily create a 2-point swing in Trump’s support.
Where do we go from here?
Nate Silver recently offered his opinion “Voting is not a problem” We agree. The problem is not the survey, but how the survey is presented and interpreted. In fact, it is remarkable that a pollster was able to speak to 800 people who agreed to be polled and get results minutes after the results of an election in which nearly 150 million votes were cast.
The problems arise because people want polls to do more than they actually can—say, decipher who’s leading a tight race or identify small shifts in a candidate’s support. The reason people think polls can do this is because polling results are often presented and discussed in ways that make them seem more accurate than is possible, often in graphs that simply show candidates separated by decimal points. These averages and graphs to the tenth point give a false picture of accuracy.
Although polling journalism in 2024 was better than it had been in the past—more coverage reported the polls’ margin of error alongside the results—much of the media discourse still gave the impression of the poll as a surgically precise tool for examining political events and campaigns. . In fact, voting is more like a butter knife than a scalpel – you can get close, but even with skill, it takes a bit of luck to get it right.
Another problem that complicates the interpretation of polling is that most pollsters avoid disclosing how they collect and adjust the data. Without knowing where the data came from — and especially how pollsters adjusted and weighted their data in hopes of trying to predict what voters would look like — it’s very difficult to evaluate or compare results. When reasonable weight decisions are made can change the margin of the query up to 8 pointsit is impossible to know to what extent the results reflect the decisions of voters, pollsters, or both.
Given these concerns and the polling errors we’ll see again in 2024, should we eliminate early voting (as Gallup and the Pew Research Center have done)? Although tempting, this is not the correct course. Understood correctly, pre-election polls can play an important role in democracy by giving the impression that results are possible. The fact that most polls showed a tight race in 2024 underscored the likelihood that either candidate could win and perhaps helped boost public acceptance of the results.
But everything needs to change. In addition to pushing for greater transparency, industry organizations such as the American Association of Public Opinion Research pushed — it’s important to be more modest about what we can learn from the survey. Polls can help determine which issues are more or less important to voters, but they will always struggle to determine winners in 1-2 point races. And in a highly polarized nation, those are often the races we have.
It is also important to be more transparent about the choices of survey participants that affect the results of the report. While survey respondents may want to focus on what they think is the best estimate based on their knowledge and skills, it seems prudent to indicate how important other reasonable options are. Given that it is impossible to know which decisions are best, it is important to know that different, reasonable decisions will produce dramatically different estimates. To see how the results change under different plausible scenarios — such as high Republican and low Democratic turnout, or vice versa — can help better convey a range of possible outcomes.
Pre-election voting is difficult. This is not an excuse – it is a reality. Treating pre-election polls as revealing deep, well-known truths without acknowledging the uncertainty inherent in these surveys risks misinterpretation, media cycles driven by bogus numbers, and a loss of public confidence in the judgment and expertise of survey and analysis participants.
Although polls in 2024 fared better than in 2020, and pollsters can safely say their polls were in the same place as the U.S. resultwe’re still asking a lot from a very blunt tool. Instead, we should think about how we can use pre-election polls to convey electoral possibilities in play and better describe uncertainties.