Philosopher Gary Gutting opines:
Public policy debates often involve appeals to results of work in social sciences like economics and sociology. For example, in his State of the Union address this year, President Obama cited a recent high-profile study to support his emphasis on evaluating teachers by their students’ test scores. The study purportedly shows that students with teachers who raise their standardized test scores are “more likely to attend college, earn higher salaries, live in better neighborhoods and save more for retirement.”
How much authority should we give to such work in our policy decisions? The question is important because media reports often seem to assume that any result presented as “scientific” has a claim to our serious attention. But this is hardly a reasonable view. There is considerable distance between, say, the confidence we should place in astronomers’ calculations of eclipses and a small marketing study suggesting that consumers prefer laundry soap in blue boxes.
Either we have to pay teachers according to test scores or not. A choice is unavoidable. Similarly, soap has to be packaged in some way, a choice is unavoidable. Better to make that choice based on research. If we can place great confidence in the research, all the better. But even if we have less confidence, so be it, because what choice do we have other than to use the research?
Cheap Talk 
4 comments
Comments feed for this article
May 24, 2012 at 9:56 pm
Bayes
Use common-sense (prior beliefs).
May 26, 2012 at 1:03 am
José Antonio Espín Sánchez
It is already an offense, and a proof of ignorance to put economists and sociologists in the same category. Some people would even been offended for putting them in the same sentence.
May 26, 2012 at 11:13 am
Jonathan Weinstein
Expanding on Mr. “Bayes,” there is some decision we would make (based on common sense or whatnot) if the research had never been performed. Whether the research is good enough to outweigh the common-sense decision can only be decided on a case-by-case basis. So it absolutely makes sense to ask about the reliability of the research. Mr. Gutting presumably thinks that people have a “pro-science bias” where they overweight an empirical study that appeared in a journal, when it has sources of error just as does common sense. I’m not taking a stand on whether he’s right, but it is certainly a cogent position.
May 28, 2012 at 6:23 am
Greg Taylor
It seems to me that a large part of the real problem is that the political-media complex has a strong incentive to exaggerate the reliability and scope of research in order to advocate their preferred policy position or sell newspapers. Contrary to Prof. Gutting’s argument, this issue is not unique to the social sciences; politicised issues within the physical realm–such as climate change or research into the causes of cancer–also attract politicians and media practitioners to strip away every assumption and qualifying caveat to prevent a highly stylised picture of research. What’s more, repeated exposure to this kind of message can only encourage the view that “…any result presented as ‘scientific’ has a claim to our serious attention.” People deserve better.