It is sometimes said that think tanks are good for democracy; indeed, the more of them the better. If there are more ideas in the public arena battling it out for your approval, then it’s more likely that the best idea will win and we will have better public policies. But intuitively, many of us have trouble believing this. We have trouble knowing who is being truthful, and we don’t know who to trust.
This battle of ideas, studies and statistics has the potential to make many of us cynical about the whole process and less trusting of all research and numbers. If a knowledgeable journalist like the Canadian Kady O’Malley expresses exasperation that think tank studies always back up “the think-tank’s existing position,” what hope is there for the rest of us? A flourishing of think tanks may let politicians off the hook, allowing them to pluck the ideas that suit their purposes and making it easier for them to justify what they wanted to do anyway.
We should put more credence on interactions that can be characterized as conversations.
Maybe we shouldn’t be surprised that think tanks produce studies confirming their (sometimes hidden) biases. After all, this is something we all do. To mitigate this, we need to arm ourselves with a certain self-awareness. Then we can be more alert to the things in a think tank’s makeup that can help us to judge its credibility. It would also help to show how public policy discussion should be structured to promote a sincere exchange of facts and ideas. Here are some thoughts about how we could do this.
1. Begin with the idea that think tanks don’t think. This is somewhat overstating it, but what I mean is that, appearances to the contrary, they are not guided by reason alone. But none of us are. Our intuitions come first, and the reasoning side of our minds serves to justify those intuitions.
Many academic economists are very quick to dismiss think tank studies because they see them as falling on the wrong side of the line between examining the economy as it is and advocating for an economy as it should be, a distinction that Milton Friedman called “positive” versus “normative” economics. In a famous essay published in 1953, the late University of Chicago economist wrote:
« Positive economics is in principle independent of any particular ethical position or normative judgments… It deals with “what is,” not with “what ought to be.” Its task is to provide a system of generalizations that can be used to make correct predictions about the consequences of any change in circumstances. Its performance is to be judged by the precision, scope, and conformity with experience of the predictions it yields. In short, positive economics is, or can be, an “objective” science. »
For serious social scientists this is an important touchstone, and dressing up normative research as positive science, inadvertently or not, is crossing a line from professional to political and is considered to entail a serious loss of credibility.
But this line is sometimes hard to draw, and in fact Friedman went on to write:
« Of course, the fact that economics deals with the interrelations of human beings, and that the investigator is himself part of the subject matter being investigated in a more intimate sense than in the physical sciences, raises special difficulties in achieving objectivity. »
The field of moral psychology has developed since he wrote this, and this field makes clear that our intuitions are automatic and instantaneous. The Righteous Mind: Why Good People Are Divided by Politics and Religion, by Jonathan Haidt, who teaches at New York University’s Stern School of Business, offers a masterful discussion of the challenges of engaging in what he calls “exploratory” thinking—searching scientifically for the truth, in the face of a strong human tendency of falling into “confirmatory” thinking—seeking evidence to support underlying intuitions.
Haidt’s first rule of thumb is “Intuitions come first, strategic reasoning second.” We are hard-wired to be that way, and Haidt’s book offers an explanation of the evolutionary advantage that our ancestors developed by evolving so that reason serves intuition. This now leads people to “bind themselves into political teams that share moral narratives. Once they accept a particular narrative, they become blind to alternative moral worlds” (p. xvi).
2. Be aware of the underlying intuition. So while we might all strive to attain Friedman’s ideal of objective, scientific inquiry, when it comes to think tanks it’s probably best to, in Haidt’s words, “keep your eye on the intuitions, and don’t take…arguments at face value” (p. xxi).
But this is a big challenge; all think tanks seem to tell us they are nonpartisan. If we understand this to mean objective, or scientific (as opposed to not being formally affiliated with a political party), we should appreciate they may well be sincere in telling us that. But that should not be enough.
Consider an exchange between Jason Clemens of the Fraser Institute and Steve Paikin, which took place during the November 11, 2014, episode of the TVO current affairs show that Paikin hosts. Paikin appears to be somewhat surprised by a representative of what, he suggests, many recognize as the most right-wing think tank in Canada. The discussion, which is part of a panel on the role of think tanks in public policy, begins after the other two panellists have placed their organizations on the political spectrum. Paikin turns to Clemens:
Paikin: “Jason, I think everybody would put a label on your organization. It’s unabashedly, they would say, right wing. Do you subscribe to that?”
Clemens: “Our approach is we are an economic think tank.”
Paikin: “So where are you on the political spectrum?”
Clemens: “We don’t worry about the political spectrum…We worry about picking issues that are important… and then using an accepted, quantitative approach to asking and answering questions. In fact, our motto here…is ‘If it matters, measure it.’ And so if you look at most of our research, it’s a measurement approach because, as we learn as researchers, you can’t argue about two plus two equalling four…Our approach is to quantify things.”
The discussion continues along these lines and there is nothing Paikin can say to get Clemens to drop, even just a bit, the veil of objectivity with which he has cloaked his organization.
In some sense this is natural; we all do it. As Haidt says, “We are indeed selfish hypocrites so skilled at putting on a show of virtue that we fool even ourselves” (p. xxii). We have no basis to question Clemens’ sincerity, but we do need an independent guide to the underlying intuitions, because we can’t expect them to be explicit.
Stephen Tapp, my co-wonk in this Policy Options issue, made our challenge a bit easier by offering an excellent application of the “If it matters, measure it” principle to uncover the underlying intuitions of Canadian think tanks. He used information on the Twitter followers of a long list of Canadian think tanks to rank them across the political spectrum. (According to his method, the behaviour of my Twitter followers would make me a “left-wing” think tank, a bit left of centre.)
It really is kind of astounding how think tank-produced ‘studies’ always end up backing the think-tank’s existing position.
—kady o’malley (@kady) August 28, 2014
A similar type of ranking of American think tanks is offered in table 1 of a 2005 paper by Tim Groseclose and Jeffrey Milyo, published in the Quarterly Journal of Economics. Their ranking is based on the number of times members of Congress cite the think tank, and they ascribe the political leanings of the members—which can be measured pretty accurately—to the think tank.
You should have maps like this in your mind, but obviously they can’t offer the whole story, as Tapp is quick to acknowledge.
We should also appreciate that think tanks may be more likely to engage in exploratory rather than confirmatory thinking, depending on how they are organized. We should also look at their organizational structure.
Haidt has the following to say on this topic:
« We should not expect individuals to produce good, open-minded, truth-seeking reasoning, particularly when self-interest or reputational concerns are in play. But if you put individuals together in the right way, such that some individuals can use their reasoning powers to disconfirm the claims of others, and all individuals feel some common bond or shared fate that allows them to interact civilly, you can create a group that ends up producing good reasoning…This is why it’s so important to have intellectual and ideological diversity within any group or institution whose goal is to find truth (such as an intelligence agency or a community of scientists) or to produce good public policy (such as a legislature or advisory board) » (p. 90)
If an organization is tightly beholden to funders, so that “self-interest” is at play, or is dominated by like-minded individuals without diversity of ideological perspectives, then it will be harder for this organization to play the game on the “positive” side of Milton Friedman’s line.
3. Lend more credence to conversation than to debate. It’s not just organizational structure that matters, but also how the interaction between think tanks is organized. Haidt points out that we are more likely to change someone’s mind if the interaction is “civil.” He suggests that “intuitions can be shaped by reasoning, especially when reasons are embedded in a friendly conversation” (p. 71).
The metaphor of public policy discussion as taking place in some sort of arena, with the gladiators fighting it out until only one is left standing, is not helpful. You should wonder about the value of debate, since this type of forum puts reputations on the line and hardens positions.
Here is an example that involves the policy discussion about the relationship between inequality and economic opportunity.
The first policy discussion is conducted as a debate, one that took place in New York City on October 22, 2014, and was organized by Intelligence Squared U.S. If you’ve never watched any of these debates, you should. The debaters are always articulate, the moderator is excellent, and the topics are provocative. But one can appreciate that the debaters must feel that their reputations are on the line; after all, they are under pressure to change the opinions of the audience members, who are polled three times, before, during and after the debate. The side that sways opinion the most is declared the winner. Gladiators in the arena, indeed!
The motion being debated was “Income inequality impairs the American dream of upward mobility.” Two teams of two debated this motion, and you don’t have to watch much of the debate to realize that there was nothing either side could say to the other that would lead them to change their mind.
This is a format, I might suggest, that in Haidt’s view hardens the participants’ reasoning in a way that reinforces their underlying intuitions. It is terribly entertaining, and informative, but it puts a premium on performance. As audience members, we may well wonder about the veracity of the facts being thrown back and forth, and at the very least we may thhink that some subtlety in interpretation could well be lost.
Contrast this format with a conversation on July 14, 2014, under the auspices of Seminars at Steamboat, on pretty well exactly the same subject. It was between two individuals who lie at opposite ends of the political spectrum: on the “right,” Charles Murray of the American Enterprise Institute, and on the “left,” Timothy Smeeding of the University of Wisconsin. At one point the moderator expresses surprise at the extent to which these two guys agree, even noting that they are wearing the same kind of shirt. After watching this I’m thinking that it would be interesting to have dinner with Charles Murray!
It is important to look at not just who is speaking but the structure in which they are placed. We should give more credence to interactions that can be characterized as conversations; conversations require leadership, goodwill and civility.
Full disclosure: I believe think tanks are valuable, and I have engaged with a number of them. Their great strength is that they offer academics things that academics so often lack: a communication strategy for research that deserves a wider audience, as well as a fountain of policy-relevant ideas that deserve more research.
Organizations, like individuals, are more likely to be open-minded and engage in exploratory thinking when they are accountable to a broad audience whose views are not fully known ahead of time and who are well-informed. So when you think about think tanks, think about who they are speaking to, and recognize that possibly, in the end, we get the think tanks we deserve.
Photo: Shutterstock
