“We were worried it was going to be a gripe site,” said the chief executive, Stephen Kaufer. “Who the heck would bother to write a review except to complain?” Instead, the average of the 50 million reviews is 3.7 stars out of five, bordering on exceptional but typical of review sites.
In fact, we can reverse the logic: “Who the heck would bother to write except to praise?”
Imagine you are asked to write a letter of recommendation for someone up for tenure. First, the university asks you if you are willing to write the letter. You mentally measure the amount of time it is going to take to read the papers. Add to that the time to write a clear and comprehensive letter. Are you going o do all that just to say something bad? Probably not. But if you are going to write something nice that gives the candidate a job for life, that might give you the satisfied buzz to counterbalance the cost of writing a letter. So, letters of recommendation will be biased towards the positive.
There is still some information: Bad candidates will get fewer letters than good candidates. Buts is this carefully noted? Is the number of letter writers who refused to write letters even recorded?
Perhaps the main countervailing force is envy. Why does X deserve tenure at highly ranked University A while I the letter writer am at humble University B? It is impossible not to write a letter for University A. If the candidate is bad, you are forthcoming. If the candidate is good, you are begrudging. But the quality of the letter is monotone in the quality of the candidate and information is aggregated. Only the very best universities are the object of envy. The rest have to decode the positive bias in their tenure procedures just like Tripadvisor users.
Cheap Talk 
9 comments
Comments feed for this article
August 29, 2011 at 10:12 pm
twicker
I think you’re pretty accurate with the university letters of recommendation.
However, with TripAdvisor, I’d suggest that the dynamics are a bit different:
First, TripAdvisor users (of which I am one) like the site; therefore, we have a bias to write reviews for the site. Now realize that, when we’re writing reviews, we’re on vacation: unless things are pretty bad, we’re in a better situation than the alternative (i.e., not being on vacation); thus, life’s just a bit rosier (or, in academese, positively biased). Sure, the hotel is just a room with some stuff – but it’s an OCEANFRONT room with some stuff!! Or a room with stuff and a FREAKIN’ AWESOME MOUNTAIN OUTSIDE!!! Even if it’s mediocre, it’s cool – because it’s located at that magical place called “Not At Home.”
Second, TripAdvisor users tend to use, well, TripAdvisor. If a place starts to get slammed, we avoid it like the plague. Where do we go? The highly-rated places, of course – which means we generally have a great time. Which means we write glowing reviews. Which means more TripAdvisor users go there, have a great time, write glowing … etc., etc., etc.
Lastly, again being part of the community (and having done some traveling), the cannier facilities will send e-mails to their happier customers to remind them to write reviews (I’d assume that they *don’t* send said e-mails to people who complain a lot). In other words, the rated establishments encourage honest, open, and overwhelmingly positive reviews by targeting who they remind. Of note, I’d be willing to bet the average review for a bed & breakfast is higher than the average review for anything else – not least because the B&B people know *exactly* who was in their house for that time, and the B&B owners actually *talk* to each individual guest and, thus, can e-mail selectively.
August 30, 2011 at 12:49 am
Dan A.
Some of this may also be cognitive dissonance at work. In general, if you pay a lot for something, like a vacation, you want to like it. What better way to prove to yourself that you liked the vacation you planned than by publicly writing a positive review and encouraging others to be as smart as you have been.
August 30, 2011 at 7:34 am
Heski
My first reaction was in line with twicker’s 3rd point – there’s some sample selection. Certainly the case for employee references (which of my managers would I ask for a reference letter).
As for the academic example I’m still on the wrong side to have a rounded view though I have some limited experience (and if you’re writing for me then do be mindful that in the light of your post and the general perception, anything less than a rave review appears to be taken as a damning indictment). My guess is that the departments are often the ones who ask for letters … and they do not ask at random. Particularly given that at least sometimes, people are told to look elsewhere before the tenure process begins, then if you’re being asked to write a letter, it may not be unreasonable to understand that as the department asking you to write a positive letter.
August 30, 2011 at 7:30 pm
Sandeep Baliga
Heski:
At Kellogg. the candidate has considerable influence over letter writers. In some cases, if the department rejects, there are no letters written for in any case. If you know the university works like that the as a letter writer you can infer that you are meant to write a positive letter.
Good luck! Of course you deserve tenure..or is that just my positive bias speaking?
August 30, 2011 at 10:19 am
Claire
Regarding tenure and the number of letters – yes, at my university, the number of refusals is recorded and used in evidence. It’s part of the strategy of picking letter writers that one tries not to pick people who might decline due to other deadlines, overwork, or who might just forget.
August 30, 2011 at 7:31 pm
Sandeep Baliga
I believe Harvard (and also Yale?) might require a huge number of letters. If you can’t raise them, game over.
August 31, 2011 at 6:07 am
Claire
I only know for sure about the promotion to Associate, but I think it’s similar for tenure. Candidate gives 6 or 7 names to the Chair, who picks 4-5 off the list to ask for letters, and additionally asks another 5 or so who aren’t on the candidate’s list. Letters from someone the candidate nominates are discounted a bit compared to the letters from the Chair’s list (since the candidate is presumed to nominate people who will write favourable letters; of course, the Chair is likely to pick people that confirm the department’s (and/orChair’s) wishes too…). The candidate’s incentive is therefore not to nominate the people most likely to write unbiased but favourable letters, but let the Chair pick those people. That assumes, however, that the Chair can identify those people, and in a small and somewhat fragmented field, that can’t always be guaranteed.
September 8, 2011 at 7:32 am
bharath
Two points
a) Firstly, I think the whole idea of Reco. letters are a BiG farse.
b) Secondly, I do not agree that Reco. letters have a positve bias. They are destined to have one because one normally seeks Reco. letter from those with whom one is comfortable with. End result — only those flowery words.
I
September 11, 2011 at 12:02 pm
The letter of recommendation - REFERENCE WORLD – REFERENCE WORLD
[…] Why Do Letters of Recommendation Have A Positive Bias? « Cheap … Imagine you are asked to write a letter of recommendation for someone up for tenure. First, the university asks you if you are willing to write the letter. You mentally measure the amount of time it is going to take to read the …https://cheaptalk.org/2011/08/2 .. […]