Apparently it’s biology and economics week for me because after Andrew Caplin finishes his fantastic series of lectures here at NU tomorrow, I am off to LA for this conference at USC on Biology, Neuroscience, and Economic Modeling.
Today Andrew was talking about the empirical foundations of dopamine as a reward system. Along the way he reminded us of an important finding about how dopamine actually works in the brain. It’s not what you would have guessed. If you take a monkey and do a Pavlovian experiment where you ring a bell and then later give him some goodies, the dopamine neurons fire not when the actual payoff comes, but instead when the bell rings. Interestingly, when you ring the bell and then don’t come through with the goods there is a dip in dopamine activity that seems to be associated with the letdown.
The theory is that dopamine responds to changes in expectations about payoffs, and not directly to the realization of those payoffs. This raises a very interesting theoretical question: why would that be Nature’s most convenient way to incentivize us? Think of Nature as the principal, you are the agent. You have decision-making authority because you know what choices are available and Nature gives you dopamine bonuses to guide you to good decisions. Can you come up with the right set of constraints on this moral hazard problem under which the optimal contract uses immediate rewards for the expectation of a good outcome rather than rewards that come later when the outcome actually obtains?
Here’s my lame first try, based on discount factors. Depending on your idiosyncratic circumstances your survival probability fluctuates, and this changes how much you discount the expectation of future rewards. Evolution can’t react to these changes. But if Nature is going to use future rewards to motivate your behavior today she is going to have to calibrate the magnitude of those incentive payments to your discount factor. The fluctuations in your discount factor make this prone to error. Immediate payments are better because they don’t require Nature to make any guesses about discounting.
Cheap Talk 
6 comments
Comments feed for this article
May 18, 2011 at 11:39 pm
Jonathan Weinstein
I think this is a side effect of an indispensable adaptation: imagination, including emotional reaction to scenarios we imagine. Being able to project possible futures contingent on different actions we might take, and evaluate those futures on an emotional level, is necessary for any sort of planning.
When the reward is just coming without any action on our part, this “early reward” isn’t doing us any evolutionary good, but it’s an inevitable side effect of imagination.
May 19, 2011 at 11:07 am
Ben
I always thought it’s a way of getting around memory/cognition constraints. A monkey could learn a bell signals food in the future, and seek out bells because he knows he likes food and the bell signifies food. But it might be easier for the brain to transfer the value to the bell as a proxy, and so the monkey seeks out the bell because the bell itself is good.
May 22, 2011 at 7:20 pm
James
Here is another attempt at explaining why Nature adopts this seemingly peculiar incentive system. There are 2 periods. The animal has to perform some action, like stalk its prey, in period 1 and period 2 in order to achieve something: get food say. The animal has a high discount factor. Dopamine in period 2 will not make the animal stalk in period 1: so nature rewards the animal in period 1. This makes the animal stalk its prey in period 1. If the animal then fails to continue stalking/get-the-meat then nature sends the dopamine levels down. It is a half-now half-later contract:although such contracts are usually motivated by a fear of the principal reneging, which can’t be the case here.
May 22, 2011 at 7:28 pm
jeff
i think there is something to this, say if there is a limited range of reward at a given moment.
March 6, 2012 at 10:35 am
Team building unlimited
Everyone is essentially distinct, nevertheless a lot of simply no not necessarily recognize this kind of. Myers-Briggs along with other personality assessments take these kinds of differences for the area and support folks …incentive maroc
December 1, 2012 at 11:13 pm
Sepp
when traveling abarod i have found that if u fly with the countries major airline its usually cheapest and they dont charge you for big bags (surf bags, fishing poles etc). when i travel central america and especially costa i always fly TACA airlines. go to there site and gauranteed its the best. they only charge $100 to change the departure date too incase you dont want the fun to end