Treatment 1 is you give people a cookie and some cake and you ask them to rate how much they like the cookie better (which of course would be negative if they like the cake better.)
Treatment 2 is you present them with the cookie and the cake and you let them choose. Then you also give them the other item and have them rate just as in treatment 1.
Of course those in treatment 2 are going to rate their chosen item higher on average than those in treatment 1. But let’s look at the overall variance in ratings. A behavioral hypothesis is that the variance is larger in treatment 2 due to cognitive dissonance. Those who expressed a preference will want to rationalize their preference an this will lead them to exaggerate their rating.
Now I wouldn’t be surprised if an experiment like that has already been done and found evidence of cognitive dissonance. The next twist will explore the effect in more detail.
The cookies will be tinged with a random quantity of some foul tasting ingredient, unknown to the subjects. Let’s think of the quantity as ranging from 0 to 100. We want to plot the quantity on the x-axis versus the rating on the y.
My hopothesis is about how this relation differs between the two treatments. At an individual level here is what I would expect to see. Consider a subject who likes cookies better. In treatment 1 he will have a continuous and decreasing curve which will cross zero at some quantity. I.e too much of the yucky stuff and he rates the cake higher.
In treatment 2 his curve will be shifted upward but only in the region where his treatment 2 rating is positive. At higher quantities the curve exactly coincides with the treatment 1 curve.
I have in mind the following theory. There is a psychic cost of convincing yourself that you like something that tastes bad. Cognitive dissonance leads you to do that. But when the cookie tastes so bad that it’s beyon your capacity to convince yourself otherwise you save yourself the psychic cost and don’t even try.
Now we won’t have such data at an individual level to see this. The challenge is to identify restrictions on the aggregate data that the hypothesis implies.