Survivorship Bias and Lessons from the EPL.

Introduction

Since I am planning to regale you with project management lessons from history, it is only fair that I advise you of some of the potential pitfalls. I am talking about our congestive biases, specifically survivorship bias.

Abraham Wald

We are indebted to a Hungarian mathematician Abraham Wald, for highlighting this bias. Wald who was born in 1902 was a mathematician with a doctorate from the University of Vienna. Like many Jews he and his family were forced to flee the persecutions of the Nazis, eventually settling in the U.S.

When working for the statistical research group of Columbia University during WWII he was asked to help the military determine how aircraft should be armored, to reduce losses.

The picture shows a representation of the battle damage data that was gathered from aircraft inspections.

The consensus was to armor where there was the greatest damage. However, Wald pointed out that the data was from the aircraft that returned from missions. What this data told Wald, was that aircraft, even though heavily damaged in those areas returned to base. The answer therefore was to armor the areas with no or little damage, because the aircraft hit there did not return. 

The problem was that the polling sample was not random, it only included the survivors.

Hence “survivorship bias”.

Lagging Indicators

Survivor bias is extremely common when one reflects on history, we often only have the information on the winners. The results of those who took similar paths but failed are ignored or never recorded.

The books “Good to Great” and  “In Search of Excellence”, were widely read and acted upon in the last two decades, but have faced  more rigorous analysis and criticism in recent years, because many of the great companies did not stay great. The data was deliberately gathered from successful companies that operated in specific ways. Comparative companies that operated in different ways were included in the study. But less successful companies that used similar tactics and applied the same strategies were not a part of the study.

When we look at successful managers, entrepreneurs, or historical figures it is tempting to say, manager “A” was super successful, so we should emulate his or her practices.

For example, Steve Jobs was by any stretch of the imagination extraordinarily successful and productive. He was also reportedly a total  a**hole at times.

Does that mean we should all be total a**holes?

If the data from numerous surveys is true, people leave managers not jobs (small “j”), which suggests that it is not a good strategy. Jobs was an exceptional individual and he succeeded despite being an a**hole at times and not because he was one.

Lessons from the English Premier League

I would like to look at one field where success and failure are openly on display and repeated annually: football managership. (soccer for those in the U.S.)

I am going to look specifically at the English Premier League (EPL), which has been in existence for 28 years.

In that time the title has been won by only 9 managers. Of the 26 titles to date 14 have been won under a Scottish manager. So!  the obvious solution is to employ a Scottish manager, or more accurately one specific Scottish manager, Sir Alex Ferguson.

FUN FACT: No English manager has won the EPL.

For more information on the individuals see FTBL CULT. https://ftblcult.com/england/premier-league-winning-managers/

Sir Alex dominated the EPL from its inception until he retired in 2013; by the perceived wisdom, studying and emulating him should bring success. Clearly that has not happened. The six completed seasons since then have seen five different managers lift the title. Now it looks like Jurgen Klopp will become the sixth in the 2019-20 season.

Manchester United have failed to maintain Ferguson’s standards of performance, despite hiring three-time title winner Jose Mourinho, who they ultimately sacked.

Chengwei Liu in his book, “Luck: A Key Idea for Business and Society”, suggests that we should look to the second level performers, rather than the superstars, if we want to emulate habits and practices. There are just so many other factors involved, including luck.

The 2015-16 winner Claudio Ranieri illustrates this point perfectly. He took the unfancied Leicester City to a dream title win, only to be fired less than a year later after a series of poor performances.

This and Manchester United’s post Ferguson performance demonstrate a well-known statistical pattern, which predicts that exceptionally good or bad performance, will likely be followed by performances closer to the mean. Interestingly several of the nine managers on the list have been sacked from their posts despite considerable other success. What succeeds at one club or time may not succeed at another.

Conclusion

The EPL of today is much changed from that of its inception, and this means that what worked in 1992 may not be successful in 2020. If emulating the skills and practices of the master was easy and successful surely the multi-billion-pound EPL would have done so.

As project managers we are also in a continuously evolving world. We too cannot rely on mimicking the successes of the past, but we can use it as a guide and a tool to expand our knowledge, develop our critical thinking, and problem-solving skills.

Therefore, we would do well to:

  • Be sure we are measuring the correct factors.
  • Having gathered the data; question if our analysis is appropriate.
  • Don’t place our hopes in superstars. (Learn lessons from the good)
  • Always remember things are continually changing and the opposition has a vote.

 

Notes:

1The original image of the B-17 was taken from a CC BY-SA 2.5 image by Emoscopes, cropped and with the red damage markers added.

This Post Has 7 Comments

  1. Superb article Jim. Really good to see this balanced approach to a subject you are so passionate and knowledgable about. Looking forward to more

  2. Though reusing an old metaphor about a compass is better than a map, with either device getting to the desired destination is what counts. In my experience, a map is quickly outdated or never complete so we pick up our compass. Our compass is composed of three freely moving segments – customer focused, team oriented, process driven. It points to the destination anytime all three segments align. Yes, there is a hierarchy and they are in the order of importance. So if they are not perfectly aligned, and they seldom are, veer in the direction of the best fit line.

    I of course realize that this comment ignores your very fine and very interesting article. But, it’s still how I try to do it. There’s a much longer explanation but I’ll reel myself in.

    Keep up the great work on this blog. It’s high enjoyable and makes me think, sometimes about the past….

  3. EL KHAYAR

    Very good article, well illustrated with exemples.
    Regarding datas, Deming said one important thing:
    “3% of the problems have figures, 97% of the problems do not.”
    I think all is said with that quote.

    1. Mark Norman

      Very good quote you cite from Deming. Thanks, I had not heard that but fits perfectly with the topic.

  4. By the way, I really like the analysis of “survivor bias” and the example given.

  5. Don GIlman

    This is all new History to me, and I effectively minored in Military History (granted with a Strategic bent) . I also build scale model airplanes. So I’m hooked!

    1. admin

      Thank Don,
      Glad you like it
      I build and paint military miniatures.
      I did not know we had those things in common

      All the best
      Jim

Leave a Reply to Mark Norman Cancel reply