Yesterday, all of us in the Good Judgment forecasting team received feedback on round 2, which has just finished. (If you have no idea what I’m talking about, please read part 1 of this series.) Time for me to reflect on the past months.
What kinds of questions were asked?
Early in the season, we were told that it would be more difficult than in the first round. This turned out to be true for three reasons. First, the admins simply asked more questions, making it harder to keep up with the tournament. Second, the share of rather obscure items was higher. Questions like the about “the removal of Traian Basescu from the office of President of Romania in a referendum vote before 1 August 2012” did not immediately ring bells with me. Third, and most importantly, the admins introduced conditional and ordered items.
Conditional items looked roughly like this: “Will Israel invade Gaza?”, (a) “if a Hamas rocket reaches Jerusalem”, (b) “if no rocket reaches Jerusalem”. While it is fairly obvious how the condition is thought to affect the probabilities in this examples, other cases were less straightforward. Anyway, this type of question further complicated the process, given that it offers another possibility to instinctively overstate probabilities, or to construct illogical connections between conditions. Some of them even had two sets of intertwined conditions.
!['Detector! What would the Bayesian statistician say if I asked him whether the--' [roll] 'I AM A NEUTRINO DETECTOR, NOT A LABYRINTH GUARD. SERIOUSLY, DID YOUR BRAIN FALL OUT?' [roll] '... yes.'](https://imgs.xkcd.com/comics/frequentists_vs_bayesians.png )