This interview was conducted on April 16 and has been published exclusively at TMV. Background on James Surowiecki and The Wisdom of Crowds — as well as a review of the book’s politically oriented twelfth chapter — are available at Central Sanity. The book can be purchased from either Amazon or Barnes & Noble. Surowiecki is also available as a guest speaker, via the Leigh Bureau.
In the three years since the book was published, what else have you learned that’s relevant to the book’s premise; anything that confirms or elaborates on the principles you outlined; anything that contradicts or raises questions about those principles?
Honestly, I’ve not seen anything that contradicts the book’s premise and that’s probably because the nature of the argument was to articulate the ways in which crowds can be intelligent, the conditions required for their intelligence: diversity, independence, and a particular form of decentralization. Those conditions haven’t changed. They still hold true.
One of the things that is new is the development or growth of institutions that are trying to figure out ways to take advantage of collective intelligence.
Prediction or decision markets, which I discuss in the book, are still underrated and underused. But there’s a lot else going on, especially on the Internet. For instance, sites like del.icio.us and flickr, which try to use the intelligence of users to construct libraries or catalogs of information, rely on the crowd’s collective judgment of what information is the most useful or relevant. Hewlett-Packard has also come up with a tool, called BRAIN, through which they pool the intelligence of a team of key executives to make forecasts and sales decisions, with good results.
Overall, I think the idea of collective wisdom has really taken hold in the last several years, and we know references to the phrase “wisdom of crowds” are growing, in blogs and elsewhere.
Also, in talking about what else I’ve learned since writing the book, I think it’s important to distinguish between two different categories of collective intelligence. The first is what you might call the “classic” wisdom of crowds – where you try to get a lot of different inputs and find a way to aggregate those inputs, like what’s done in the stock market or at the race track; like Google does on search. The second category is when you cast a net widely and tap a big group, but what you’re really looking for is the idea of one person; you’re trying to find the one person in the crowd who has the “right” answer – like what’s done on Yahoo Answers. Those two categories are related, but they’re not identical, and only the first one was really the subject of my book.
Finally, I’d say that across all of these devices, if you’re really interested in the wisdom of crowds, you need to watch out for compromising the ability of people to think for themselves. You need to make it harder for them to look at what everyone else is doing. You need to protect their independence. That’s one of the hurdles that certain sites on the Internet run into. The things that make them fun and engaging could compromise their users’ independence. Like Digg, which is a great site. But when you decide how to cast your vote, you know how everyone else is voting. That needs to be watched, if you really want the decisions to be as intelligent, as accurate as possible. Otherwise, you might be feeding into mob mentality, which is obviously what Charles Mackay worried about.
What about the conundrum of applying the wisdom-of-crowds to social and political issues? You touch on that application briefly in Chapter 12 of the book; but again, do you have anything new to add to the discussion, anything that you’ve come across in the last few years? Anything new, for instance, on the use of “deliberative polling”?
That’s a great question, but I’m not sure I have a great answer. Cass Sunstein, a law professor at the University of Chicago, has done a lot of interesting work on juries and deliberation, which he talks about in Infotopia, where he suggests deliberation might actually make groups of people dumber rather than smarter.
So I think the value of deliberative polling remains an open question. But there are other ways that the wisdom of crowds is being tapped in political and social contexts. For instance, there are communities or community organizations that are trying to do a better job of incorporating the local knowledge of citizens into their work on issues like transportation. In Britain, social agencies are trying to more regularly poll their “customers,” the people who are the actual recipients of social services. What would those recipients say about the administration of benefits if they were engaged in the process; if they were asked what works and what doesn’t?
Many of these efforts are more informal than formal, but at least the administrators of social programs are recognizing that there’s a lot of knowledge among “consumers of systems” and that knowledge should be tapped.
How is that work different than the consumer polling and focus groups that the titans of brand management, like Procter & Gamble, have been pursuing for decades?
It’s basically the same thing. What’s different is that, in government and social work, and maybe in the medical community – those are not areas where this type of exercise has traditionally been undertaken. Of course, [when they do undertake it,] their challenge is the same [as the consumer marketer’s challenge]: The wisdom of crowds works best when the members of a group generally agree on what a “good answer” is or should look like. In politics, the views are so different, the differences of opinion so great. And there’s the paradox: For crowds to be smart, you want differences or diversity of opinion, but you also want general agreement about the goals you’re trying to reach.
One other thought on this topic, going back to market research [by the likes of P&G]. Most of that research asks consumers, “What would you do?” [What product would you choose?] But the real gains in this research will come when organizations start asking large groups questions like, “Do you think this product will be a success?” – in other words, when they start seeking individuals’ thoughts about how other individuals will react or behave [like in a prediction market or betting pool].
BrainJuicer, a UK company, has done a series of tests in this way, in forecasting consumer behavior. The work is in its early stages, but they’ve had some good results so far. And that research has proven much cheaper than traditional market research because there’s no filtering involved. Instead, they rely on self-selecting groups.
Earlier, you mentioned the three conditions for the crowd to reach a smart conclusion: diversity, independence, decentralization. You also noted, in cases like Digg, that the independence factor is or could be compromised. How might organizations – in the private or public sectors – mitigate that effect and protect the independence of the members of the group?
I want to be clear on this. Independence is necessary, but it doesn’t have to be as strict as some might think or as I might have seemed to suggest earlier. You don’t need people living in a vacuum before the crowd can reach a smart conclusion. The real key to making a deliberative group work in a smart way is diversity. Sunstein talks about this. Genuinely diverse groups, with real disagreements among them and not a lot of consensus to start with – those groups tend to become smarter, especially on factual questions, and ultimately more moderate during the course of their deliberations. In contrast, if the group is not truly diverse, it tends to become dumber as the members deliberate. Of course, it is harder to get people to work together in truly diverse groups. Regardless, diversity, and the disagreement that goes with it, remain critical to reaching smart conclusions.
I don’t recall you ranking the conditions in the book. So let me make sure I understand you. If you’re saying diversity is the most important condition, how would you rank the other two conditions? Is independence the least important?
Diversity is definitely the most important condition for the crowd to reach smart conclusions. Independence is crucial, but you don’t need perfect independence for the group to thrive. Decentralization is really a method for obtaining diversity.
The other key to crowd wisdom, of course, is the effective aggregation of crowd inputs. In a democracy, that basically means voting. However, as you noted in your book, political issues are not cognitive problems because political issues don’t have a standard that defines a right versus wrong answer. Instead, political issues are more like cooperation and coordination problems, which don’t necessarily have a right or wrong answer either. But real solutions to those problems are still possible, and those solutions tend to emerge over time, through a process of constant revision or iteration. Does that suggest we should have more frequent voting on more issues, allowing us to “iterate” our way to better political solutions; to better cooperation and coordination solutions?
That’s a more interesting question than it probably gets credit for [being]. People often view voting as a “fake solution.” And there are certainly big problems with voting in this country: Low participation. Voter doubts that their vote can really make a difference, etc. Regardless, with all the tools and mechanisms available to us, tools like those we talked about earlier that can help us aggregate knowledge from more people on a more regular basis, it would seem there must be a way to make this concept of more frequent voting work, and I have to think there’d be value in doing so.
Of course, another problem we run into when thinking about this are the voting “systems” [or voting methods] that we currently us. They’re very crude. Ideally, you don’t just want people to vote “yes” or “no.” What you really want is a sense of intensity of preference, a sense of ranking of choices. The best kinds of judgments are those that are “probabilistic” (involving probability). We likely won’t see that anytime soon in political voting – but where you can measure intensity of preference you tend to get a better, more reliable, more accurate result.
One place such an approach might be useful would be when we already have rough agreement of “what” we want to do, so the open questions are not about “what” but about “how.” In those situations, the wisdom of crowds [captured through iterative voting] could be valuable. For instance, national or homeland security. We all agree we want to keep the country safe. That’s the “what,” leaving us free to tackle the “how” – “how” we coordinate and/or cooperate to reach that goal.
In addition to writing for The Moderate Voice, I have my own blog. In the early days of that blog, I tried a “wisdom of crowds experiment” on a social issue and ended up, instead, getting into an interesting debate with one of my readers, who essentially argued that democracy works, not because crowd wisdom leads to good decisions, but because dissipation of power limits bad outcomes. Do you think that’s right – that democracy’s value is not found in the wisdom of crowds; rather, it’s simply the lesser of evils, better than a dictatorship or oligarchy?
No. It’s more than just a situation of what’s less bad. True, limiting the amount of power in the hands of a single person or small group is a good thing. Larger, more diverse groups make it less likely that we end up with terrible decisions. Our democracy recognizes that. And we reached that decision, as a nation, collectively. We (the crowd) deserve some credit for that. Making our leaders have some accountability, some responsibility to the people they’re leading is a good testament, I think, to the wisdom of crowds.