The election campaign in Germany is about to gather speed with less than 30 days left until election day. I assume we’re going to cover that in more depth soon, too. For now, I can direct you to the Hertie School’s Expert Blog on the German Federal Elections in 2013 in case you have not checked it out yet. They cover a plethora of topics from labour market policies and the German Energiewende to gender equality and family policy.
So let me do the kick-off for some posts that will appear on this blog over the next couple of weeks. Yet, this is not about politics but looks at polling and forecasting in the German case, thereby briefly touching upon some of the recent trends of the German political landscape. I will point out some of the flaws of both polling and forecasts. However, don’t misread the point: I’m not against polls and forecasts as such and have lots of fun following the respective discussion throughout the year. But we should not overemphasize these results, either, as both do not come without problems.
Forecasts
Let’s start with the forecasting. This is not very prominent in the German political landscape. Of the top of my head, I’m only able to name two forecasting models, which are not that different in the end (but I’m no specialist in this area of expertise so please add any other model you are aware of in the comments section). Next to the chancellor model by Norpoth and Gschwend, my current favorite example I like to direct people to and which has stirred some discussion has recently been published in the blogosphere. Mark Kayser and Arndt Leininger have come up with a forecast called the benchmarking model. They are predicting that
the current coalition of CDU/CSU and FDP will receive a vote share of 47.05% on September 22nd. Taking into account the stochastic nature of predictions and that the likely vote share needed to obtain a governing majority in parliament will be around 45.5% we also calculate the probability for the current coalition to stay in power: 83.18 percent.
Their model is based on four variables: previous election results, voter identification with the incumbent government, the number of terms the government has been in power, and the relative economic performance of Germany compared to the three other most important economies in Europe (France, the UK and Italy; what they call “benchmarked” growth).
To be honest, I do not think this is the best forecast. It is a frequentist model for the inherently Bayesian problem of forecasting voting behavior. Also, Kayser and Leininger are only interested in predicting how well the incumbent government is doing and leave aside all the other parties. But this was not my choice to be made, so let’s look at some critique regarding the benchmarking model.
Kayser and Leininger claim that “economic growth trumps campaigning” based on the title of their blog post as well as the benchmarking model’s name: “Implicitly by using the deviation of German growth rates from the average of British, French and Italian growth, we are presuming that voters judge the state of the economy relative to that of other countries.” It is not about growth itself but about relative growth compared to other countries. And it is basically about the people’s perception whether the economic performance is acceptable, but we do not have data for that.
Yet, leaving these quibbles aside, I have two concerns regarding their main argument. I am willing to buy that economic factors – however slight – are favoring the incumbent government. My concern is rather what economic factors we should look at. Growth? Poverty? Unemployment? Growth is certainly not the only relevant (economic) indicator.
Second, I’m nevertheless not convinced that campaigning does not matter at all and I’m wondering what data their argument is based upon. You can’t infer the influence of a campaign from the outcome. A victory does not mean that the winner’s campaign was all too perfect, nor that the losing party’s campaign was a tremendous disaster. There is enough evidence suggesting otherwise (here, here, here, but also the great scholarly work on campaigning). So, I would rather be interested to what extent economic performance hampers campaigning or whether campaigning influences the perception of the state of the economy, but I do not see how economic growth actually trumps campaigning.
In addition to that, they claim not to include any current polling results in their forecast. So, I assume that Kayser and Leininger would have also published this forecast claiming that the incumbent government will likely have another term in office two years ago as well, shortly after the Fukushima incident. Really? (see polling results below for Germany’s political situation back in April 2011 and compare ). I just don’t buy that this forecast, in the way it is set up now, would have been published if all polls had predicted otherwise. Leininger and Kayser’s forecast went live on August 6 when most of the German poll agencies have come up with the recent trend that the current CDU / CSU / FDP coalition government (re-)gains popularity. Having this information certainly added to their confidence in the model’s predictive power.
Nevertheless, go and check it out. It is very well written and one of the very few attempts actually taking a shot at forecasting in Germany. Given the huge attention Nate Silver and other forecasters received in the last US presidential elections, I was hoping to see more forecasts during the German federal elections campaign. But there is none, sadly. (Any guesses why?)
Polls
Part of the German political life is the traditional Sonntagsfrage: “If the federal elections would take place next Sunday, which of the following parties would you cast your vote for?” Different institutes produce these polls, often by orders of TV stations or newspapers. They use different methods and most of the time they do not come to the same results. Often, they are accompanied by questions regarding the reputation of the political personnel or regarding current political topics (Germany’s nuclear power phase-out, NSA surveillance).
Throughout 2013, the polls have been very stable (as can also be seen here). Angela Merkel’s CDU will most likely win the popular vote and five parties will end up in the parliament (CDU / CSU, SPD, Green party, Left party; with the FDP being the uncertain prospect, but I’m almost sure that they will end up above the 5% threshold).
One should keep in mind that these polls only ask for the vote intention and represent only a snapshot in time. Polls are a nice way to illustrate the political atmosphere but their weight is limited given the non-event problem (most of the time there will not be an election next Sunday). Hence, they also tend to take up much noise instead of making a reliable prediction. While these polls are usually produced on a monthly basis, the frequency increases before elections and, by now, a new poll is published almost every day. Observers value those last rounds of polls published shortly before elections because they are said to more or less predict the election outcome. But this often the object of some debate, also in the US.
With every poll comes the problem of assembling and weighting a sample, the challenge of having a representative sample, and the problem of false answers. German polling agencies do not publish the raw data but weigh the results in their respective models. But these are, of course, very well-kept secrets and we do not know the distribution of vote intention in the sample as expressed in the responses.
Furthermore, polling results have a rather bad reputation in Germany based on their historic records. The agencies often miss the election outcome by large margins as portrayed for some of these agencies for the last four federal elections. Two cases can serve as anecdotal evidence in this regard. In 2005, all major agencies predicted a landslide victory of the conservative party and Angela Merkel would become the first female chancellor in Germany. The CDU /CSU would have been able to initiate a coalition government with the liberals according to these results, their favorite partner back then. Yet, all agencies overestimated the CDU / CSU result by 6 to 7 percent. Angela Merkel nevertheless seized power, but in a grand coalition with the social democratic party. The second example refers to the Liberal’s (FDP) surprising election result in the state elections of Lower Saxony in early 2013. All the polls predicted that the FDP will have a hard time to enter the parliament let alone renew the coalition government with the conservatives. Eventually, they ended up in the parliament with almost 10% of the popular vote, but could not stay in power since the Social Democrats and the Green Party could secure the majority of the seats in the parliament by a small margin.
So, what should we take from that?
Forecasting is not very prominent in Germany and the models are far away from the very sophisticated stuff that is going on in the US. However, I am very curious to see how the benchmarking model will do in the long run (next two to three elections) and whether it will do better compared to the predictions of the German pollsters.
It is almost common sense that the conservative CDU / CSU will lead the field followed by the SPD and the Green Party. Yet, polls do not tell us much about the government that will be formed after election day. If either of the two camps (CDU/CSU and FDP vs. SPD and the Green party) outperforms the other camp, the problem of government formation will not come up. But this depends also on the election results of the other parties, whether they enter the parliament as well as voter turnout – all of which are not or only partially covered by the benchmarking model or by the pollsters. Including all these factors makes it harder to predict the outcome, but it makes it all the more interesting at the same time.
Thank you for this very interesting post (and the ping to my blog^^).
While I am still preparing the write-up for the upcoming season, there is one point I would like to argue a bit here.
Furthermore, polling results have a rather bad reputation in Germany based on their historic records. The agencies often miss the election outcome by large margins as portrayed for some of these agencies for the last four federal elections.
While the sentiment is definitely there and while the examples you listed are accurate, I think it is perhaps a bit unfair to conclude that polling is altogether “bad” in Germany. For three out of the last four federal elections, the eventual result (coalition possibilities) was accurately predicted. The election in Lower Saxony was the first time in the history of polls that an established non-extremist party previously consistently rated below 5% managed to significantly beat their polls. For every example of this, there are tons where the FDP (or the Green, or the Left) arrived at the predicted 3%.
Other surprises in recent polling history include the sudden rise of the Pirate party for the elections in the regional parliament of Berlin, which was not seen in the polls for the simple reason that it only occured during the last days before the election, well ahead of the usual latency the polls have.
Of course it is those times that we remember a lot, also because politicians and media tend to cite the cases frequently if it befits the current narrative. But there is a plethora of, from a demoscopic perspective, boring elections as well (like the last time in Thuringia or Hamburg) without any hickups.
These are just my thoughts, of course, and as always, we will only know about the 2013 poll quality after September 22nd.
Kind regards, Burtchen.
Fair enough, referring to a ‘bad reputation’ may be a bit dramatic. Nevertheless, I would still argue that this is not only a matter of wording, but that there is a real sense of distrust among the German population, media commentators as well as (at least some of) the political scientist around me when it comes to polling results. And this distrust is not only about the predictive power of polling results but also relates to the way the results are presented. See Stefan Niggemeier’s take on Forsa as well as Forsa’s supposed bias against the Social Democratic Party (both in German).
Look. I don’t think that the polls are bad, but their reputation / perception is bad; probably because the way the polls are presented is not good either. It is true that most of the time the polling agencies publish results that are more or less correct (within the margin of error of 3% for big parties and 1.5% to 2% for small parties). Nothing is wrong with missing the actual result once in a while since sometimes one just misses the correct result. They just should tell the average consumer what their results actually mean.
[Btw note: The margin of errors refer to big and small parties, i.e. parties around either 40% or around 10%. But the landscape of political parties has changed over the last couple of years in Germany. To name but the most obvious changes, we witness 5-party-parliaments as the rule instead of an exception, decreasing electoral turnout, and changing distributions of votes or the diversification of voting behavior.]
In the end, my punch line should have been read as follows: both forecasts and polls come with flaws and we should not overemphasize their respective value, BUT should not underemphasize them either. Just for a real good forecast, I would argue that one would need to integrate both long-time effects as well as constantly updated polls.
Additionally, there are the problems of biased results and different effects on voting behavior. I didn’t want to dive into this but since I already started the point why not also address these issues. Let’s look at the results for those examples that I have mentioned in the post and the one you added in your comment.
For the federal elections in 2005, both the CDU / CSU and the FDP polling results did not match the actual election result. The FDP side of the story is often forgotten in this mess. *All* polling agencies’ results were biased in that they overestimated the strength of both parties. The results should have been within a range of +/- 3.0% for the CDU / CSU and within a range of +/- 1.4% to 1.6% for the FDP (different margin of errors for bigger and smaller parties, see here in German). Nevertheless, it is stunning that all of the polls were biased over a certain period of time. Any explanation for that? Have the pollsters told us how this bias came about?
Regarding the state elections in Lower Saxony and Berlin, the picture is a little different.
In Lower Saxony, polling agencies supposedly estimated the ‘correct’ vote intention some days before the election, but then the “the CDU base should lend their votes to the liberals in order to save the government” campaign kicked in. Or, maybe the voters thought it would be a pity when the liberals would not be represented in the parliament (polsci parlance: underdog effect).
In Berlin, the success of the Pirate Party is in part due to the media campaign evolving around their initial popular support. So, when some people admitted to intend to vote for the Pirates, the media took up this story and many more people realized that there would be the chance that the Pirates will enter the parliament. (polsci parlance: bandwagon effect)
Predicting these kinds of behavior is probably impossible. But that only shows again that one should take polls for what they are: polls. They do not predict an outcome, they take up much noise, and they sometimes are not able to take up recent trends. That also should tell us that whatever the polls are saying 30 days before an election does not determine the outcome. And campaigning, TV debates, and all the other activities that are going to happen will have an impact on the election outcome.
Another forecasting model is done by Kai Arzheimer using the Bayesian framework:
http://www.kai-arzheimer.com/polling-german-election-polls-how-certain-is-merkels-victory
Very interesting, thank you!