If you are programming a robot to vacuum your floors here’s one thing you would never consider doing: endow the robot with feelings of happiness and sadness and teach it to be happy when the floor is clean and unhappy when it is dirty.
But evolution led us to a state of affairs where emotions are what motivate us to do our jobs. How could such a kludge arrive.
Here is a story. Primitive organisms are reproduction machines. They need a certain chemical in the environment, and when they can obtain that fuel they can reproduce.
So the most successful primitive organisms are those that are the best at finding fuel. Natural selection favors those that seek fuel.
Next the organisms get more complicated. They have to make decisions that involve more than just immediate reproduction. They have intertemporal tradeoffs, multi-dimensional consumption, etc.
There is infrastructure in place to simplify this transition. The organisms that have survived to this stage are the organisms that seek fuel. They have built and learned systems for doing what is necessary to get fuel. So fuel is a simple and effective incentive mechanism.
The organism could evolve a mechanism for storing and later releasing fuel. Fuel is released when the organism takes certain actions. Fuel-seeking organisms will take those actions. Natural selection will favor the organisms that release fuel for the right actions.
Now remember that fuel is the energy needed for the most primitive functions of the organism. When this fuel is released the organism gets a boost of that energy.
A boost of energy is a big part of what we call happiness.
(subject to the usual disclaimer, this is based on some conversations with Balasz Szentes.)