| |
| |
| |
| |
| |
| |
| |
| |
| The Art, Science, and Power of the Poll | |
| |
| |
| Most people think of me as a political pollster, and rightly so. Much of my public profile is tied to politics, especially during presidential and congressional campaigns. But politics makes up less than a quarter of the work I do. The overwhelming majority of my professional time is devoted to measuring and interpreting public opinion for corporations and other business interests and for professional organizations. While voters get to vote only once or twice a year, consumers vote with their wallets every day. | |
| |
| |
| Besides, as different as they might seem, political and consumer polling are pretty much the same thing. In both instances, we make choices based not just on price and value, or promises and policy, but on unconscious signals that we receive and interpret to satisfy our unconscious selves. Business leaders and politicians often miss this essential point. Our minds think in similes and metaphors-we search for comparisons with which we are comfortable to help us understand the unknown. To get to this deeper level of decision- making, good survey research goes beyond simply asking respondents if they prefer Product A or B, or Candidate C over Candidate D. | |
| |
| |
| Good research has to include creative questions that tease out the public's deeper values and identities, and the questions themselves need to avoid whenever possible charged phrasing that can badly skew responses. That's particularly true with political polling. As George Lakoff shows so effectively in Don't Think of an Elephant, controlling the language on key issues gives a party a big leg up in controlling voter response. If we had asked a survey sampling in, say, the spring of 2007 whether the United States should "cut and run" in Iraq, the results would have been far more negative-and far less reflective of true public opinion-than if we had asked the same question in neutral language that included a phrase such as "troop withdrawal." On the other hand, if the aim had been to manufacture positive numbers for the president, "cut and run" would have been just the words to use. | |
| |
| |
| The point is that asking questions is only the beginning of good polling: The way you ask them, the language you use, and the effort you make to broaden the connection with those being surveyed all determine the value and ultimately the accuracy of the collective response. Only policy wonks have opinions about HR Bill 313, but if HR Bill 313 happens to concern, say, the quality of drinking water in exurban communities, just about everyone has an opinion on that. The challenge is to put a question in terms people can understand and react to without losing the reason for asking the question in the first place. | |
| |
| |
| An example: On the Saturday before the November 2000 presidential election, I inserted for the first time into our daily survey of four hundred likely voters the following question as a way of leap- frogging past the horse race aspects of the contest to the underlying motivation of voters: You live in the land of Oz and there is an important election for mayor this year. The candidates are the Tin Man, who is all brains and no heart, and the Scarecrow, who is all heart and no brains. If the election were held today, for whom would you vote? | |
| |
| |
| We could have simply asked voters whether on Election Day they were more likely to cast their ballot for someone who was highly intelligent or someone who was highly empathetic. By then, the stereotypes were well-established: Al Gore was a master of policy and governmental detail, but he was wooden in a crowd and on the debating stage. George W. Bush, by contrast, was widely perceived to be loosey- goosey on detail work, sometimes tongue-tied in his responses, but empathetic to the heart's core. On the surface, the choice was simple: Do you want a president who was the smartest guy in the class or one who feels your pain? But that's also the problem with basing questions on stereotypes: They reduce what should be complex answers down to emotional responses to a few simple catchphrases. | |
| |
| |
| Instead, we framed the question in terms of what are almost archetypal figures in American popular culture: the Tin Man and the Scarecrow. Respondents for the most part knew which stood for which candidate, but now they had an image and a narrative framework within which to consider their choice. They weren't choosing solely between brains and empathy; they were choosing between two characters found along L. Frank Baum's famous yellow brick road. By phrasing the question as we did, we both simplified the choice and added complexity. Which candidate did voters want to shepherd Dorothy on her journey through Oz? And which one did they want for themselves, waiting, somewhere over the rainbow, when the election was over? | |
| |
| |
| That, at least, is what I was hoping when I came up with the question, and events, as it turned out, agreed. When our results came in on the Sunday before the election, after asking the same question for three days, the precision of the tie-46.2% to 46.2%-told me we were not going to know the winner on Election Day. (Truth in packaging: I didn't reveal this part of the equation when I told the vice president Monday evening that we wouldn't have a victor the next day.) Our seemingly trivial question got at the fundamental image both candidates projected. But the question was about so much more than heart and brain, and about so much more than presidential politics; it was about the soul of Oz itself. Was that fictional world-and was our real one-to be ruled by reason or compassion, by love or by policy? And in the end what the question revealed was both how conflicted and divided we Americans were at the start of the new millennium. | |
| |
| |
| That divide would soon get codified into the artificial construct that came to be known as "red states versus blue states"-artificial because a swing of a few hundred votes in either New Mexico or Florida would have turned those red states blue and handed the presidency to Al Gore. I would see both the divide and conflict in starker relief in my own values polling-polls, for example, that tracked how bitterly separated Americans were on the question of abortion, at the same time that they showed how many of those who opposed late-term abortions believed fervently in a woman's right to choose. And, of course, we would hear and see the divide and the conflict amplified and exploited in the run-up to the 2004 presidential election. But the Oz question is where I saw it first, in unmistakable form, as if someone had cleaved the electorate exactly in two. | |
| |
| |
| All this from one seemingly innocuous metaphorical inquiry that worked so well because it fit the situation and the people so exactly. When I tried the same question four years later, the Tin Man won by 10 percentage points, a big enough margin to suggest a John Kerry victory over George W. Bush, but as our subsequent analysis showed, by 2004 the premise of the question was no longer valid. The president's performance post-9/11 had convinced many voters that he had brains as well as a heart. In the wake of what appeared, initially at least, to be a well-reasoned and forceful response to the attack on the American homeland, Bush's approval rating hit 85 percent in my poll-and as high as 95 in some others-and stayed in the high sixties for well over a year. For Kerry's part, he didn't project pure intellect the way Gore had in 2000. In the end, we realized that we had pulled a question off the shelf and tried to recycle it without first thinking through whether the comparisons still fit. | |
| |
| |
| Here's another example from the world of politics that illustrates what happens if you jump to conclusions and don't focus closely enough on the details. You might recall that on the afternoon of the 1992 New Hampshire primary, CNN caused a great stir when it hinted at and then projected that the underdog conservative commentator Pat Buchanan was going to embarrass the incumbent President Bush when all the votes were counted. To its great embarrassment, CNN got that one wrong, but it wasn't until I got involved in New Hampshire polling four years later that I understood why. | |
| |
| |
| My natural instinct has always been to poll throughout the day, in the run-up to the vote and as people vote. Doing so just makes sense to me: That's how you make sure that everyone is heard from-the late shift and the early one, night owls and day people. All-day calling also leaves you time to redial the people you missed the first time around. Back then, though, the idea ran against conventional wisdom. Polling was basically a two-tier affair: exit surveys earlier in the day and dinner-hour calling when everyone was presumed to be back home. In fact, my upstart round-the-clock polling was so controversial that ABC News refused to air my results. I knew I was right, though, and, sure enough, come primary-election day, Buchan- an's voters once again showed up disproportionately early at the polls and skewed the numbers for other pollsters who were relying on traditional methods to make their projections. | |
| |
| |
| I saw the same thing in the 2000 primaries with the younger Bush and John McCain, and in New Hampshire again in 2004 with Howard Dean and John Kerry. In state after state, McCain would be down a few points to Bush, only to come surging back after five or six o'clock. In New Hampshire, Dean appeared to have picked up so much momentum by midafternoon that NBC News was about to wring my neck for insisting the race was still close, but I knew Kerry's voters were going to come out in droves late in the afternoon and in early evening. We had tons of pre-election data showing his numbers spiked after five p.m., and that is exactly what happened. That's why, to this day, we annoy the public by calling all day long. | |
| |
| |
| From Red State vs. Blue State to Wal-Mart vs. Macy's: Where We Shop Is How We Vote | |
| |
| |
| Annoying, in fact, is probably what pollsters do best, but until you ask the question, you can't find that kernel of truth, and until you chew the kernel, you can't begin to know what's inside. A few years back, I made a big splash by identifying a new conservative political majority emerging among Wal-Mart shoppers. Wal-Mart? What does it have to do with majority anything, other than majority shopping? Quite a lot, actually. Where we shop says a lot about how we vote, who we admire, and what we believe in. | |
| |
| |
| Our polling shows that weekly Wal-Mart shoppers, about a fifth of all those who shop at the store, are far more likely than those who never shop at the retail giant to: be Hispanic, live in a rural area, attend church at least once a week, and-the greatest point of distinction-identify themselves as either conservative or very conservative. In 2004, when John Kerry lost the popular vote by only 3 percent, he lost among weekly Wal-Mart shoppers by a whopping 76 percent to 24 percent. Meanwhile, those who told us they "never" shop at Wal-Mart went just the opposite direction, voting 80 percent for Kerry and 18 percent for Bush. | |
| |
| |
| Even when President Bush's popularity began to slide generally with the populace, it held firm with Wal-Mart's core shopper base. In 2005, when only 44 percent of Americans retained a favorable impression of the president, 65 percent of the retail giant's most frequent shoppers told us they approved of the way he was leading the nation. Starting in the summer of 2005, all that began to change. As the president's approval ratings were slipping into the thirties with the electorate generally, he gradually fell under 50 percent with frequent Wal-Mart shoppers and ultimately into the low forties. To me, that was a far more telling indicator than any other polling numbers that Bush was losing the American people-including much of his own core. | |
| |
| |
| "Retail politics," in fact, can be taken literally as well as figuratively. If you want to carry your message to liberals, think Filene's with a liberal-conservative ratio of 51-29, Bloomingdale's (48-26), Macy's (42-32), Neiman Marcus (39-30), and Target (39-36). After Wal-Mart, conservatives prefer Sears (16-57), JCPenney (21-50), Kohl's (23-50), Boscov's (26-53), and Kaufmann's (29-46), which has since been bought by Macy's. If it's a perfect balance you're seeking, Marshalls is the place. I found that 34 percent of Marshalls customers lean liberal and 35 percent conservative, with the rest undecided or independent. These political alignments with particular stores are so consistent that I've come to think of retail locations as a cluster of mini-precincts-and these mini-precincts are very definitive marketing opportunities. | |
| |
| |
| Shopping destinations, it turns out, are equally as useful as predictors of political leanings as the red state-blue state paradigm and ultimately may prove more effective. I have a far easier time envisioning a "red" state such as Virginia or Colorado or Florida going "blue" in 2008-or a "blue" one like New York going "red" had Rudy Giuliani gotten the Republican nomination-than I do Wal-Mart voters deserting the GOP despite their current disenchantment with George W. Bush. Indeed, I have a not-so-far-fetched vision of a time in the near future when election night TV maps will be peppered with store logos, and instead of swooning over which way Ohio votes, we'll swoon over which way Target and Kohl's have fallen. I have an even clearer vision of candidates making media buys not through TV or radio stations, but through store catalogs. Why not take the ad where the swing voters are shopping? | |
| |
| |
| Equally, the political leanings and related sociological background of store shoppers tell us enormous amounts about what will move off the shelves and which products will sit forever. Remember when Wal- Mart tried to launch its designer label line? The thinking was obvious-there's no point losing sales to label-conscious shoppers when volume guarantees you can undercut the competition's price-and it was obviously wrong. People don't shop at Wal-Mart out of snob appeal. That's for limousine liberals and Wall Street conservatives. They shop there for the breadth of offering (everything from foodstuffs to prescriptions to ammo and beach balls) and because Wal- Mart knocks down underwear that normally costs $5.97 to $2.95. Forget that populist appeal, and you've forgotten everything that matters. So definitive are these retail affinities and loyalties that when Macy's purchased the midwestern Kaufmann's chain, I found myself wondering if the New York-based retailer was as interested in broadening its ideological base as it was in strengthening a thin regional presence. | |
| |
| |
| From Red Skelton to Richard Pryor: What a Good Laugh Says | |
| |
| |
| As a pollster, I believe in casting the broadest possible net because you never know where an answer might be hiding, and I believe in parsing the data that comes in to the nth degree and looking for unusual connections, because sometimes it's in the strange crevices where interesting truths begin to emerge. A case in point: Back in December 2006, in surveying for AOL, we threw in almost as an afterthought the following item | |