The problem with polling young American voters

Unlock Editor’s Digest for free

Over the past six weeks, depending on which poll you look at, U.S. President Joe Biden has held a lead ranging from 28 points ahead of Donald Trump with those under 30 to a 21-point deficit. If realized in November, the first solution would represent an improved margin with young voters compared to 2020. The second would constitute a stunning reversal and would surely propel Trump back into the White House.

To state the obvious, one of these figures – or more likely both – does not give an accurate picture of the opinions of young people today. But why are they so different?

One factor is simple: Opinion polls have margins of error that widen as the demographic you examine shrinks. Most people are familiar with the margin of plus or minus three percentage points that accompanies a typical survey of 1,000 people. When we focus on those under 30, which sometimes have less than 100 respondents, this uncertainty can reach more or less 10 points.

But in this case, we are faced with a gap of more than 50 points. The variation is also much greater among those under 30 than among older groups, and even more so than in the past. Clearly, there’s a lot more going on beneath the surface.

As the notorious example of the 1936 Literary Digest poll, which confidently predicted Franklin D. Roosevelt’s defeat in that year’s presidential election (he won in a landslide), shows us, the quality of the sample matters more than its size. Their huge error was based on a poll of more than 2 million American voters.

Today, all reputable pollsters carefully recruit, sample, and weight respondents to ensure that initial results reflect the correct number of young people, women, college graduates, etc., making it impossible to fail on a scale so humiliating. But what is true for a survey as a whole is generally not true for its subgroups. Pollsters generally don’t guarantee that there are a sufficient number of Latinos, for example, or high school dropouts among those under 30.

A recent analysis by Lakshya Jain and colleagues at the US political analysis collective Split Ticket illustrates the problem. They surveyed several hundred Americans ages 18 to 29 using different sampling, interviewing, and weighting methods, and found that three different approaches—all of which are used today by different traditional pollsters—gave very different results. These ranged from +18 for Trump when young adults were not weighted to be internally representative, to +16 for Biden with carefully calibrated weights and sample.

Chart showing that different survey methodologies can produce very different results

The fact that pollsters don’t consistently weight demographic subgroups like this isn’t an oversight — it’s simply not what nationally representative polls are designed to do. And as Natalie Jackson, vice president of US market research firm GQR, points out, weighting small samples can do as much harm as good. So when political enthusiasts or participants “dive into the crosstabs” to examine results for particular demographics, they are looking at data that is not meant to be representative of those groups.

This means that in one survey, those under 30 might be more male, while in another there might be more college graduates, etc. This alone can explain much of the variation we see this year.

Additionally, the old challenge (trying to find young people who will answer the phone) is replaced by a new one. Pew Research found evidence suggesting that a significant number of people who register for online polls give false information, and that this is especially true for those claiming to be young voters.

But all is not lost. A regular survey that bucks the trends described above is the Harvard Youth Poll, which surveys 2,000 young people under the age of 30, whom they recruit using official records. It then rates them to be representative of American young adults as a whole in terms of age, race and education, income, region and language.

The results are almost exactly the same as the similarly sampled and weighted version of Split Ticket – a solid double-digit lead for Biden, albeit diminished from the 2020 result. Which suggests that a large portion not not only from the wide variation, but also from the The apparent shift to the right among those under 30 observed elsewhere could be the result of unrepresentative samples of young voters.

None of this is to say that Democrats should be complacent when it comes to supporting young people, especially amid the unrest over the Gaza conflict. But the alarming picture painted by some polls of a pro-Trump surge among young people is almost certainly a mirage.

[email protected], @jburnmurdoch