Statistician Andrew Gelman had a sense that something was up with dubious Science article soon after it was published.
As you probably know, the well-known Science article on attitudes toward gay marriage by LaCour and Green has been called into question (even by its second author) and will likely be retracted by the journal.
We were amazed and impressed to learn today that statistician Andrew Gelman had a sense that something was up with the article soon after it was published. In a December comment in the Washington Post, Gelman was flabbergasted by the size of the claimed result:
What stunned me about these results was not just the effect itself—although I agree that it’s interesting in any case—but the size of the observed differences. They’re huge: an immediate effect of 0.4 on a five-point scale and, after nine months, an effect of 0.8.
A difference of 0.8 on a five-point scale … wow! You rarely see this sort of thing. Just do the math. On a 1-5 scale, the maximum theoretically possible change would be 4. But, considering that lots of people are already at “4” or “5” on the scale, it’s hard to imagine an average change of more than 2. And that would be massive. So we’re talking about a causal effect that’s a full 40% of what is pretty much the maximum change imaginable. Wow, indeed.