Public policy debates often involve appeals to results of work in social sciences like economics and sociology. For example, in his State of the Union address this year, President Obama cited a recent high-profile study to support his emphasis on evaluating teachers by their students’ test scores. The study purportedly shows that students with teachers who raise their standardized test scores are “more likely to attend college, earn higher salaries, live in better neighborhoods and save more for retirement.”
How much authority should we give to such work in our policy decisions? The question is important because media reports often seem to assume that any result presented as “scientific” has a claim to our serious attention. But this is hardly a reasonable view. There is considerable distance between, say, the confidence we should place in astronomers’ calculations of eclipses and a small marketing study suggesting that consumers prefer laundry soap in blue boxes.
Either we have to pay teachers according to test scores or not. A choice is unavoidable. Similarly, soap has to be packaged in some way, a choice is unavoidable. Better to make that choice based on research. If we can place great confidence in the research, all the better. But even if we have less confidence, so be it, because what choice do we have other than to use the research?