Friday, March 5, 2010

Overconfidence And Anchoring

Noting that positive (negative) earnings surprises tend to be followed by more positive (negative) earnings surprises, economist Dr. Hersh Shefrin makes the point that this occurs because the investor/analyst tends to be overconfident, and furthermore fails to fully incorporate new information (i.e. he is "anchored" to his previous opinion).

To demonstrate how these two respective tendencies work, Hersh discusses two experiments where subjects are asked to provide answers to the following:

1) The Dow Jones Price Index does not include the effects of dividend re-investment. If dividends had been considered re-invested in the index since its inception in 1896, at what price level would the index be at today? Provide a 90% confidence interval around your answer (i.e. you are 90% confident that your interval includes the right answer).

2) There are 100 bags, each containing 1000 poker chips. 45 bags have 700 black chips and 300 red chips, while 55 bags have 700 red chips and 300 black chips. If you select a bag, what is the probability that most of the chips are black? If you pulled out 12 chips from that bag, and 8 of them are black and 4 of them are read, now what is the probability that most of the chips in the bag are black?

The answer to the first question is around 650,000, which lies far above the upper limit of the confidence interval of most subjects of whom this question is asked. If humans were well-calibrated, the number should have been inside the confidence interval of 90% of subjects. Instead, this experiment demonstrates that humans are over-confident, believing they know the answer (within a margin) when they don't, and believing they are more skilled than they are (e.g. 90% of car owners believe they are better-than-average drivers).

The answer to the first part of the second question is 45%, which most people get. What people fail to do is incorporate the new information (namely, that 8 of the 12 chips pulled out are black) successfully to determine the new probability required in the second question. While most guesses are between 45% and 75%, the real answer is north of 96%! Shefrin argues that people aren't even close in their answers because humans tend to anchor themselves to their previous opinions, and are unable to objectively incorporate new information.

This combination of overconfidence and anchoring leads to positive (and negative) surprises in earnings results that persist despite the presence of new information. This provides opportunities for investors who can overcome this bias. Needless to say, Shefrin is not one of those academics that believes in the Efficient Market Hypothesis.

Disclosure: Author did not answer either question correctly


Unknown said...

I am trying to figure out how to calculate the answer to the second part of question two - how do you reach the 96% probability?

Saj Karsan said...

Hi luis,

tatimatla has graciously provided an eloquent solution as follows:

It's a wonderful illustration of the Baye's theorem.
Define two events A and B as:
A: Selecting a bag that's got predominantly black chips.
B: Drawing 12 chips and ending up with 8 black and 4 red chips.

It's our job to find out P(A given B).
Now P(A given B) = P(A or B)/P(B) = P(B given A)*P(A)/P(B)

We're left to find out the values of each of the individual probabilities that go into the above equation.

The easiest first -> P(A) = 0.45

P(B given A) is nothing but landing with 8 "successes" (defined as the event where a black chip is obtained in the selection) with the probability of "success" being 0.7.
This equals 12C8 * 0.7^8 * 0.3^4 = 0.23114

P(B) = P(B given A)*P(A) + P(B given A')*P(A') = 0.23114*0.45 + [12C8* 0.3^8 * 0.7^4]*0.55 = 0.10832

Hence, P(A given B) = 0.45*0.23114/0.10832 = 96.04%