News, Reflections and Ideas

Rare Events and Why Averages can be Very Misleading

7 September 2022Kevin Keasey and Charlie Cai

Introduction

This blog discusses a phenomenon we are subject to throughout our lives, and it may be on a daily basis, without fully realising its consequences. For a number of reasons, the exceptional tends to be talked about and gains the attraction of the media, etc. The normal run of the mill is not usually reported and is rarely discussed. Just consider the stories we tell our friends – they tend to be focused on the exceptional and unusual because this is what people want to hear. They don’t want to listen to you droning on about how you did the washing that morning, or what you bought at the supermarket on your last visit.


However, while we talk about exceptional, rare events because they are interesting, the exceptional is then seen as the norm because this is all that is talked about – the distribution of our talking points is the exceptional and these form our benchmarks for comparison. This form of comparison is dangerous in all walks of life (just consider the problems being caused by youngsters making comparisons on social media) and it is no different in the financial world.


In the next section, we discuss the influence of rare events on our perceptions and then in the second section we focus on why the notion of averages can distort analysis in the context of rare events.


In both sections, we will use examples from all walks of life to show how the phenomenon of ‘rare event’ distortion is so ubiquitous.


Rare Events are ‘Everywhere’ and we Do Not Notice


Humanity itself is a rare event. There are billions of stars and as far as we know, we are the only example of intelligent life. But we see humanity as the norm and not the exception. We do not consider that the people we share our lives with are the outcome of an extremely rare event. Similarly, as individuals, we are the outcome of two individuals coming together and the probability of any specific coupling is a rare event. These types of events shape our lives – the meeting of partners, the choice of career, the meeting of key individuals in your life, etc. If you look back on your life and consider how many different choices/turns it could have taken – you will soon realise that the particular life path you have experienced is a rare event. Given the possible number of choices you could have made at each key ‘turning point’, you could have ended up with 1,000s of different life paths. However, we don’t really appreciate that any particular life is likely to be special because of the combinations of the choices made.


So, whatever we see is likely to be the ‘special outcome’ of a series of decisions made through time. The failed business may be the outcome of meeting the wrong business partner, it may be the outcome of expanding at the wrong time, it may be because of a choice of the sector at the wrong time, it may be the outcome of any combination of these and other decisions.


Let’s consider how some of the above applies to the business and financial sectors. When we see a successful small company, we don’t fully appreciate the low probabilities of survival. About half of small companies will die within the first 5 years and another 20 to 25% will die in the next five years. So only 25% are likely to survive past the 10-year mark. Of these small firms, only a fraction will grow to a significant size and an even smaller fraction will list on a public share market. If we consider there are about 5 million companies in the UK at any one time, when we focus on an FTSE 100 company, we are looking at a 1 in 50,000 example!


Most examples of success are rare events in business life. Venture capitalists are fully aware of these odds and they take a portfolio approach to their investments. They know that a lot (between 50 and 60%) will lose money, while only 1% will make 10x returns. They know they will have to undertake on average 100 investments to find an exceptional one. The same applies to projects and staff. We see the successful projects and the stellar staff, with failures and average performance being barely noted. Few companies seem to take the time to undertake post project audits, with failed projects just being ‘placed on the shelf’ and largely forgotten.


Focusing on the successes and not discussing the very common failures also applies to investment gurus. We know about Warren Buffett’s successes but it is difficult to gain much information on his failures, and the same applies to all investors. They will tell you, ad nauseam, about their big wins but are conspicuously quiet about their losses.


Even when we turn to consider the ‘survival’ of large companies, we are blinded by the longevity of a few, well-known household names – BP, Shell, Rolls Royce, etc. But when you start digging into the typical longevity of large companies, you will be surprised.


The tribulations of Blackberry, Yahoo, etc. should remind us that the large companies of today are not the same ones of the past. Creative destruction (Schumpeter) is still alive and well. A study by Mckinsey found that the average life span of companies listed on the S&P 500 had declined from 61 years in 1958 to 18 years today. The same report believes that up to 75% of the companies on the S&P 500 will have disappeared (sold, merged or failed) within a 5 to 8-year horizon. This loss of large companies is down to them becoming very inefficient, complex and difficult to manage – most of us will have experienced, as staff and/or customers, the staggering inefficiencies of large organisations. In short, they have to spend more time managing themselves than serving their customers.


So, success is rare but often talked about, poor performance and a lack of drive is a lot more typical but not usually a topic of lasting conversation (though the state of the UK presently is fuelling a lot of discussion of Broken Britain).


What does all of the above mean for investment? Well, rare events are what we focus on and these become our ‘benchmark’ distributions. This distortion is added to when we are sold particular notions of average performance – we discuss these next.



Why Averages are Miselading in Finance


Average values are often thought of as typical or the most common values. But this is not always strictly correct from a statistical perspective. Let’s consider male life expectancy in the UK as an example. What we know from general reading/experience is that most people die in old age, though there is still quite a high level of childhood mortality. This leads to a very heavily right skewed distribution where the mean (the average value of the distribution) is less than the median (the value that separates the distribution into two sets of an equal number of observations) which is less than the mode (this is the most frequent number in the series – this more akin to the notion of typical). When talking about life expectancy, there is a convention to talk about the mean, which is 79 years for males in the UK. But this figure has been brought down by the rate of childhood mortality. What is more relevant to most people is the mode of the distribution – i.e. what you can typically expect if you survive childhood – this is about 87 years in the UK for males. This is quite a large difference and should make a big impact on retirement, especially given the ‘onslaught’ of health issues in the 80s. For completeness, the median is about 82.5 years.


Why the above is important is that the pension industry tends to use the mean age when discussing life expectancy and this gives a very distorted view of what most of us will have to plan for and deal with. What they should be discussing is what most adults can typically expect – and here it is the mode of 87 years. Similar problems occur in finance and when looking at the returns investors can expect on average. A lot of the issues here are common to distributions in general.


The first thing to realise is that many distributions are prone to extreme values (outliers) and these can hugely distort mean values. Consider the example of five children – four of them are 3 foot tall but one is already a giant at 8 foot. The mean value would suggest that the children are, on average 4 foot tall – a foot taller (33% taller) than is the actual, typical value. The same can happen to portfolios of stocks when one or two (Amazon, Microsoft historically) stocks can hugely inflate the mean return. And here we need to take account that the Amazons of this world are rare events. To be conservative, it would be more appropriate to look at either the median or the mode of the distribution. This is especially important as it is well recognised that stock distributions are ‘fat-tailed’ – meaning there are a number of observations in the extremes of the distribution – if they are equally balanced in the negative and positive tails, then this will no matter, but if we have more in the positive tail, then this will give a false picture of what can be typically achieved.


The second issue is that while a distribution describes historic observations, how far such distributions have predictive power is quite another thing. This is especially true if the distribution has rare events – the probability of rare events (in the tails) being repeated is, by definition, disappearingly small. And here we reach the nub of this blog.


Rare events are likely to have distorted historic mean returns but trying to predict such rare events going forward is incredibly difficult. This is why historic mean returns can be quite misleading. What would be more appropriate from a predictive stance would be the typical returns that could be earned – and here, we should look at the modal and/or median returns. This approach might be considered highly conservative but it will give a ‘truer’ picture of what can typically be achieved and whether the ‘mental costs’ of being invested are truly worth it. For example, there are many periods when the modal returns are not overly impressive and it is questionable whether it is worth being invested. The only way of really dealing with the vicissitudes of stock returns is to stay invested for a long time, ride the losses and hope for the gains. Just remember, that it is the successes that are always talked about (failure and averages do not receive the same airtime) and mean returns are often inflated by hard-to-predict rare events. If only the life of a stock investor could be as simple and rewarding as portrayed by the investment industry (the sunlit uplands gain) but sadly, it rarely is!



Idea – Revisiting indexing


Given the disproportional impact of rare events on our investment returns (both good and bad) and our emotions (feeling extremely lucky or regretful), ideally one would like to avoid the worse and ride on the best ones. The next best would be able to stay neutral (obtaining a return that is less affected by these extremes, using median and mode to evaluate the expected return).

The challenge is that we cannot know which company will become the next big thing or disaster. For the disaster, diversification would have taken care of it and the largest downside is a 100% loss (when not leveraged). The fear of missing out (known as FOMO) on the 10x or 100x increase seems an even bigger concern. While the probability of repeating a rare event is low for a given company and, therefore, its past extreme may not be repeated, at a cross-sectional level, a new future rear event (another Google, Amazon or Tesla) will occur as investors shift their valuation to favour new technology or a business model breakthrough. So the simple investment strategy is to try to include those positive rare events in your portfolio. Hence the long-run indexing strategy has become increasingly an attractive answer since John Bogle’s revolutionary innovation of index mutual funds in the mid-1970s. While the academic argument for indexing has been focusing on the market efficiency hypothesis, the importance of selecting those successful outliers into your portfolio cannot be ignored.

In the following, we show the 5-year cumulative monthly return of portfolios of new securities added to S&P500. We skip the first month to make sure these returns are accessible. The cumulative 5-year return is 86.35% on average the median level of return is slightly lower at 63.55%. These are strong returns especially when we cumulate over a long period.



If we compare this index-inclusion strategy with the total return of S&P 500 indexing, the annualized median 12-month return is 14.7.9% for this strategy which is higher than the SPX index return of 11.29%). The compound of this would mean an addition of 8351% over 35 years.

However, one last note about the importance of exceptions. The US market itself is not an average case. Not all passive indexing will produce equal results. In the last several decades the US has been the centre of corporate innovation and produced many world-leading companies. The passive indexing in this market would work well in this market with a compound return of over 41 times. All other leading world indexes have a considerable gap in their performance when compared to the S&P500. Given the well-known home country bias in investors’ portfolios, it will not be obvious that 35 years ago that an ordinary European would consider putting a significant part of their asset allocation in a foreign index. In this regard, even a passive strategy would require some level of activeness to choose a market.

In those markets, investors need to work harder to capture the exception. It requires a relatively more active approach and, there, is still no guarantee. One smart-passive approach is to use quantitative approaches that are based on company fundamentals to capture good quality companies. As the Buffet-inspired screening described in “The Experts and The Evidence” book, we show below that active stock selections would be able to significantly improve the investment return compared to the passive index investing in the main benchmark index of the FTSE100.

(This is not investment advice. It is for information and discussion purposes only. If you like to know more about the detailed setup of the tests mentioned above, please email us at charliexcai@gmail.com).