If a tree falls in a forest and no one is around to hear it, does it make a sound?
This old philosophical conundrum can be mapped into the dilemma facing the aging academic:
If I publish a paper and nobody reads it, teaches it or cites it, can it ever be a truly great paper?
As with all questions with no Platonic certitude, economists say: Let the market speak and tell us the answer.
Glenn Ellison has studied a more serious version of my question in his paper “How Does the Market Use Citation Data? The Hirsch Index in Economics.” The Hirsch index for an author is the highest number h such that the author has h papers with at least h citations. So, an index of 5 means you have five papers with at least five citations and that you do not have six papers with at least six citations etc.
Glenn points out that the Hirsch index doesn’t do a great job at ranking economists. Nobel prize winner Roger Myerson’s Hirsch index is a mere 32. But he has a few papers with over a thousand citations. Seminal papers in economics tend to get a huge number of citations but most only get a few. So, the plain vanilla Hirsch index needs to be re-evaluated.
Glenn turns to the market to guide his measure. He studies an index of the form h is the highest number such that the author has at least h papers with at least a times h to the power b citations. The plain vanilla Hirsch index sets a=b=1. Glenn estimates a and b in various ways. In one method, he looks at the NRC department rankings and finds the variables a and b that best predict the NRC rank of a (young) economist’s department. To cut a long story short, a=5 and b=2 come out as the best predictors. With this estimation in hand, we can perform various comparisons – Which fields are highly cited? Which economists are highly cited? Etc..
Here are some tasty morsels of information. International finance, trade and behavioral economics are highly cited fields (Table 6). Micro theory and cross-sectional econometrics are the worst and IO does not do too well either. These facts mean Yale and NU, which are strong in these three areas, are under-cited economics departments. But basically one gets the picture that an economists citations are closely connected to the rank of the university where s/he is employed.
Ranking young economists, it is pretty obvious who is going to come out on top: Daron Acemoglu with an index of 7.84 (Table 7). This means Daron has 7.84 papers with roughly 300 citations. Ed Glaeser and Chad Jones are close behind. Once you adjust by field, more theorists start to rank highly: Glenn, Ilya Segal, Stephen Morris and Susan Athey pop up. Also, my friend Aviv Nevo gets a shout out as an underplaced guy.
A few comments:
Most of these people are tenured well before their citations go crazy. Expert opinion not data-mining leads to their tenure. This tells you how well expert opinion predicts citations. Also, to the extent that citations take time, expert opinion will always play a role in tenure decisions. There is a difference between external opinion and internal opinion. The same few people always get asked to write letters and they will do a good job. But internal opinions may be more noisy and depend on the quality of the department. Then, Glenn’s field-adjusted citation measure gives you some idea of a candidate’s quality and might be a valuable input into the tenure decision.
Finally, there are citations and citations. A paper getting regular cites in top journals is better than a paper getting cites in lower tier journals. This can be dealt with by improving the citation index.
At another extreme, some papers may be journalistic, not academic, and then their citations mean less. For example, Malcom Gladwell gets high citations for the Tipping Point but he did not do any of the original scientific research on which his book is based. Of course he writes wonderfully and comes up with amazing examples and he is clearly an intellectual. I bet Harvard would love to have him an as an adjunct professor but they will not give him a tenured professorship.
Despite these caveats, the generalized Hirsch index is an interesting input for academic decision-making.
Cheap Talk 
1 comment
Comments feed for this article
October 11, 2010 at 3:22 pm
pauloabx
I’m not sure that from the fact that economists from some fields are more highly cited, you can actually say that the field is more highly cited. To see this notice that economists who do theory tend to write more solo papers, while, for instance, behavioral economists tend to co-author a lot more.
So imagine that every paper is cited 100 times. If three theoretical economists write 1 paper each, they’ll all have only 1 paper with 100 citations. If three behavioral economists co-author three papers and all have 100 citations, then the field is equally cited and they all have higher indexes.