The underperformance is even worse over the long term - over 15 years more than 92 per cent of funds investing in American shares underperformed their respective benchmark. The same sad story is repeated around the world and in Australia more than 70 per cent of funds investing in Australian shares underperformed their respective indices in the past 12 months.
All of this may be a little surprising to local retail investors because, according to many NZ financial advisers and fund managers, there are lots of good reasons why passive investing is "dumb". These reasons range from the silly to the stupid and include :
•"The higher the fees you pay the higher will be your return"
•"You should only buy a passive fund if you don't know which active funds will outperform"
•"Successful investing is an art not a science"
Over the years this column has highlighted various research papers by academics which demolish those arguments and the "successful investing is an art not a science" nonsense is revealed as such by the fact that computers, which rely on science, consistently outperform most fund managers.
In fact, in light of the Standard and Poors research, it would be more correct to say that "unsuccessful investing is an art not a science".
But there is no arguing with the huge flow of funds away from active and whilst passive investing might be "dumb" institutional investors like the NZ Super Fund are not.
What hasn't had much attention is why fund managers find it so hard to beat the index -
that is, owning everything and going to the beach.
Fund managers aren't stupid.
Disingenuous, greedy and unethical on occasion, but stupid, no.
Indeed, until recently there was concern that too many bright young minds in the US aspired to do a finance degree at Harvard or Yale and get super rich via fund management or investment banking at the expense of the engineering and medical disciplines.
So why is it so hard to beat a mechanical process of just owning everything on the stock market?
In February 2017 two pieces of research were published in the US illustrating, in relatively easy to understand terms, why beating the index is so difficult.
The first paper, entitled "Why Indexing Works" by JB Heaton, a lawyer, and Nick Polson and Jan Witte, of the University of Chicago and the University of Oxford respectively, was reported on Bloomberg under the heading "The Math Of Futility".
This alludes to what the authors regard as the mostly futile attempts to beat the index.
Bloomberg summed up the findings of the paper as follows : "The distribution of returns in the stockmarket is bizarrely lopsided.
Often equity benchmarks are so reliant on gigantic gains in just a handful of stocks that missing them - as most managers do - consigns the majority to futility."
Bloomberg used an example to make the point : "Say you have five poker chips, four worth $10 and one worth $100. The five chips have an average value of $28 but what if you reach into the bag and pull out two $10 chips repeatedly? That's roughly how managed funds approach stocks. The problem is the majority of selections will fail to snag the $100 chip. Mathematically there is an average value of $56 across 10 two-chip combinations - the problem is six out of 10 times you will grab a pair with a sum of $20. The same things happens with stocks - only a few managers will own the winners."
Bloomberg adds that that example even overstates the probability of picking a winning stock.
The second paper, "Do Stocks Outperform Treasury Bills?", by Professor H. Bessembinder of Arizona State University, shows how infrequent that $100 chip is in a stock market context. His research makes fascinating reading and it should be included in Finance 101.
The key points of the paper are that :
•Even though the average of the stock market outperforms short -term government bonds most individual shares do not - 58 per cent of all US stocks listed on the US stock exchanges between 1926 and 2015 underperformed short-term government bonds over their full lifetimes.
•The reason the share market outperforms bonds is due to a small number of outperforming stocks. The entire gain in the US stock market since 1926 is attributable to the outstanding performance of the best-performing 4 per cent of listed stocks.
In the paper Bessembinder explains that the reason the stock market outperforms bonds in aggregate but the average stock underperforms is because some stocks perform extraordinarily well.
In statistics this is known as positive skewness.
His analysis shows that about 26,000 shares have been listed on the various US stock exchanges since 1926 and total lifetime shareholder wealth creation to December 2015 was US$32t.
But of the 26,000 stocks only the top 1000 performers account for all of the US$32t in wealth creation.
The performance of the other 96 per cent of stocks only matched that of short-term government bonds.
So how is this analysis relevant to a retail investor based in Auckland NZ?
The key lesson is that if you can't pick the winning stocks - and let's face it, not many people can - the best way of ensuring you own the top performers is by owning an index fund that owns everything.
The common practice in NZ of owning just five stocks in NZ, five in Australia and five in global markets is thus a recipe for disaster.
Don't be fooled by the stories arguing that successful investment is an art not a science frequently accompanied by a picture of Warren Buffett, the godfather of active management, smiling down benevolently.
Buffett is an exception to the rule and in recent years even he has found it difficult to beat the index.
Brent Sheather is an Authorised Financial Adviser and a personal finance and investments writer.
