Why I Hate Polls and the Reporting of Polls
So here’s the crux of the WaPo headline generated by editors looking at the results of a joint poll commissioned by their employer and ABC News: “Confidence in Obama reaches new low.”
Contributing writers at Chris Cillizza’s WaPo blog describe it as “an erosion of confidence” in the President.
While I’m no Nate Silver, I’m pretty sure these same editors and writers could have studied the details of the polling results and reached a different conclusion, namely: “Despite economy, oil spill, confidence in Obama effectively same as six months ago.”
How so? My logic: First, I went to the WaPo’s data detail page, and scrolled down until I found the relevant question: “3. How much confidence do you have in [ITEM] to make the right decisions for the country’s future – a great deal of confidence, a good amount, just some or none at all?”
From there, I scrolled down a little more until I found not just the most current results, but the related trending data for President Obama, going back to January 2009. There, in that data, you can clearly see that — yes — comparing the month of his inauguration to the latest results, there has been an “erosion in confidence.” However, comparing January of this year to July of this year, you see a trendline where there has been very little change. The sum of “great” and “good” confidence was at 47 in January 2010 and is at 43 now, barely outside the survey’s 3.5-point margin of error.
This entire four-point drop can be traced not to those with “great” confidence — they were at 24% in January and are at 24% now — but to those with “good” confidence, at 23% in January and 19% now. And where did those four points of “good” go? They split: two points went to the “none” (i.e., “no confidence”) column and two points went to the “just some” column.
That strikes me as rather remarkable. That strikes me as the news. After six months — during which even Obama supporters like me have questioned his leadership ability during the early weeks of the Gulf Oil crisis and doubted his support for more deficit spending — this President has held virtually even, losing only two percentage points to the “no confidence” column.
This is why I hate polls and the reporting of polls. Because they too often tempt us to reach overly simplistic conclusions.