Last week I attended a lecture by Stephen Walt on Why simplistic hypothesis testing is bad for International Relations.1 The lecture took place at the College of William and Mary, Williamsburg (VA) were I am currently residing as a visiting scholar. Walt’s talk was based on an article that he co-authored with John Mearsheimer for the EJIR special issue on The End of IR Theory, 2013. While Walt made some convincing arguments about the steady increase of non-theory driven big data analysis and how they change IR as a discipline, I was rather nauseated by his unreflected conception of theorizing as virtue.
Walt and Mearsheimer’s main claim is that IR is lost in a spiral of ever more quantitative and ever more correlational research. Small questions are answered with tailor-made datasets that are used to test as many variables as possible. Those tests are not, as they should be, guided by theories or indeed used to create new or refine existing theories, but lead to disconnected empirical findings. Debates these days are fought over who can build the biggest datasets and generate the most powerful models rather than who can come up with the best explanations for the ‘big picture’ of international relations. It has become so much easier to test correlations – and get published on it – that few of us (and young scholars in particular) are motivated enough to sit down and think in hypothetical terms anymore. In short: IR has lost its interest in grand theorizing.
Empirically speaking, I think Walt and Mearsheimer have a point here and the number of large-N correlational studies has indeed increased over the past years. I also agree that bad data and the excessive use of mentioned bad data in mindless hypothesis testing is bad. However, I was absolutely stunned by the authors’ simplistic (reference to their own title intended) definition and glorification of theories.
“Theories (…) are like maps. Both aim to simplify a complex reality so we can grasp it better. A highway map of the United States, for example, might include major cities, roads, rivers, mountains, and lakes. But it would leave out many less prominent features, such as individual trees, buildings, or the rivets on the Golden Gate Bridge. Like a theory, a map is an abridged version of reality.“
Theories are meant to simplify complex realities? Point taken. But their production process and use is just as political as that of maps: rivers might be where rivers are, but the definition of what and where South and North are, the decision of what is at the map’s center and what at its margins, and the need to fix borders on paper, are conscious actions. They are taken by people and have an impact that goes well beyond the paper the map is written on. What bothers me, therefore, is the authors’ ignorance – or active neglect – of this political dimension to their analogy. What I think about this is best illustrated by commenting on the eight “virtues of theories” according to Walt and Mearsheimer:2
“1. Theories provide overarching frameworks — the ‘big picture’ — of what is happening in myriad realms of activity. There is simply no way to understand an infinitely complex world just by collecting facts.”
So far so good. However, Walt and Mearsheimer actively excluded “non-positivist” theories like critical theory, feminism, and “other constructivist” theories, including some parts of constructivism itself, from their observations.3 That more or less leaves us with realism, liberalism and mainstream constructivism. In other words: he excluded all approaches that aim to question the smaller parts of ‘what is happening in myriad realms of activity’ in a theoretical sense and focus on those theories that aim for the ‘big picture.’ By doing this they reinforce their ‘good simplistic theories’ vs. ‘bad simplistic hypothesis testing’ dichotomy.
“2. Powerful theories can revolutionize our thinking.”
Again, I agree. Question is: who decides when a theory is powerful? From the American perspective realist explanations for the Cold War might indeed have been very powerful. But for the rest of the world? Especially for those countries for which the war actually was not that cold, deterrence and balance of power were probably not as high on the list of best explanations for current events. But since the American academy (and with it big parts of IR mainstream in other countries) used to dominate the discourse, alternative approaches have had hard times to be regarded as powerful.
“3. Theory enables prediction, which is essential for the conduct of our daily lives, for policymaking, and for advancing social science.”
Whose daily lives? Whose policymaking? Whose social science?
“4. Theory is essential for diagnosing policy problems and making policy decisions.”
This is one of the most stunning parts of Walt and Mearsheimer’s logic: the loss of theorizing is the reason for the widening gap between (US) academia and (US) policy. A number of research projects have testified for the existence of such a gap (including the TRIP project), but they usually blame the tendency of IR scholars to formulate too abstract theories for it, rather than its excessive supply of data. What is more is that they thereby indirectly admit theories’ role as state ideology’s sidekick. If policy making should not be based on empirical findings, but rather be guided by assumptions and core ideas of IR theories then scholars find themselves either to reflect a state’s or government’s current needs and demands for ideology in their theories or to risk losing their “powerful” label all together.
“5. Theory is crucial for effective policy evaluation. A good theory identifies indicators we can use to determine whether a particular initiative is working, because criteria for evaluation are embedded within it.”
If all of the above and following is taken into consideration, my opinion about using theories as the ‘objective’ yardsticks of policy evaluation should be obvious. I do not say that this is not done, nor that it per se should not be done, but it is far off of being an innocent virtue. The power dimension in this becomes even more apparent when considering that some theories (“the positivists”) are more likely to be used for such evaluations than others.
“6. Theory enables us to look at the past in different ways and better understand our history.”
At one point during the talk Walt bashed the simplistic-hypothesis-testing camp for their attempts to measure something far off like the German-Franco war while it is hard enough to gather reliable data for wars as recent as Iraq. I agree. But why then should it be possible to explain past events, about which apparently we know so little, with contemporary theories? If theories are indeed like maps then it would be rather foolish to use 20th century maps (of the US) to find your ways through 19th century France. To do so would assume that IR theories are independent of time and space, which even Walt and Mearsheimer admit is not the case.3
“7. Theory is especially helpful when facts are sparse.”
I agree that it can be helpful and necessary to think beyond what facts can tell us. And at this point I could just as well criticize the idea of ‘facts.’ But my main point is that all that we spin together after data stopped being accessible and are making sense, is a conscious decision for going with one explanation or another. We do have different theories – of which Walt and Mearsheimer acknowledge only two-and-a-half – and they would all pick different routes even if (!) they could agree on a common interpretation of the ‘facts’ available up to that point. Different actors are important, other modes of action considered to be at play, and more. All of this diversity is ok, but please do not tell me about the virtue of theories as a whole if you are so clearly convinced that only one theory is right about all of this.4
“8. Theory is critical for conducting valid empirical tests.”
Sloppily collected data is bad? Agree. Incoherent hypotheses are bad? Agree. Overambitious kitchen sink hypothesis testing is bad? Agree. All hypotheses that do not directly relate to some of the grand theories are bad? Disagree. The authors do not refer to hypotheses generating research, but to theory-informed – aka not simplistic – hypothesis testing. Question is, if we continue to start out from the same theories again and again, how are we expected to think outside of their handcrafted boxes?
Do not get me wrong, I actually agree that theories and theorizing are important. And I do by no means want to defend the ‘virtues’ and objectivity of data. But we should all keep in mind that even the most objective sounding explanations – theories and correlational hypotheses both – are social constructs of (groups of) scholars, and potentially their political allies. They are meant to make sense of the world as they see it or would like to see it. That does not mean that all theories and theorists are categorically evil, but they should be understood as what they are and where they come from. IR as a discipline is at a point were theories from other parts of the world will soon rival with the transatlantic mainstream, just as the countries they originate from are becoming counterweights on the international stage. And here again, we should not neglect their intellectual roots in politics and ideology. To close with Cox: “Theory is always for someone and for some purpose. All theories have a perspective. Perspectives derive from a position in time and space, specifically social and political time and space.”6
1 Mearsheimer, John J. and Stephen Walt (2013): Leaving theory behind: Why simplistic hypothesis testing is bad for International Relations. European Journal for International Relations, Vol. 19(3), 431.
2 Ibs.: 435-437.
3 He formulated this exclusion slightly different in the talk than in the paper. I could debate their definition of a constructivist vs. positivist dichotomy at this point but I spare you that.
4 NB: Walt Mearsheimer are both prominent advocates of realism. Just like Walt reassured the audience that their evaluation of the situation is not motivated by their own conviction, I could claim that my knowing about their theoretical preference does not influence my own evaluation. But that would go a bit against my original purpose for this piece, wouldn’t it?
5 Mearsheimer and Walt: 432.
6 Cox, R. (1981): Social Forces and World Orders: Beyond International Relations Theory, Millennium, Vol. 10(2), 128.