After a disaster happens the post-mortem investigation invariably turns up evidence of early warning signs that weren’t acted upon.  There is a natural tendency for an observer to “second-guess,” to project his knowledge of what happened ex post into the information of the decision-maker ex ante.  The effects are studied in this paper by Kristof Madarasz.

To illustrate the consequences of such exaggeration, consider a medical example. A radiologist recommends a treatment based on a noisy radiograph. Suppose radiologists differ in ability; the best ones hardly ever miss a tumor when its visible on the X-ray, bad ones often do. After the treatment is adopted, an evaluator reviews the case to learn about the radiologist’s competence. By observing outcomes, evaluators naturally have access to information that was not available ex-ante; in that interim medical outcomes are realized and new X-rays might have been ordered. A biased evaluator thinks as if such ex-post information had also been available ex-ante. A small tumor is typically difficult to spot on an initial X-ray, but once the location of a major tumor is known, all radiologists have a much  betterchance of finding the small one on the original X-ray. In this manner, by projecting information, the evaluator becomes too surprised observing a failure and interprets success too much to be the norm. It follows that she underestimates the radiologistís competence on average.

The paper studies how a decision-maker who anticipates this effect practices “defensive” information production ex ante, for example being too quick to carry out additional tests that substitute for the evaluator’s information (a biopsy in the medical example) and too reluctant to carry out tests that magnify it.

A tip of the boss of the plains to Nageeb Ali.