Geoff Gannon August 11, 2019

How Safe Can You Really Make a 5-Stock Portfolio?

By GEOFF GANNON

Investors often overestimate the reduction in volatility they will get from diversification and underestimate the reduction in volatility they will get from simply owning stocks with a beta less than 1.

 

Over the last 10-11 years, I’ve owned 5 or fewer stocks in about 90%+ of all quarters. My portfolio’s returns have had a lower standard deviation in terms of returns than the S&P 500. And in terms of just “downside volatility” – which is what most investors mean when they say volatility (they aren’t bothered by big moves to the upside – only the downside) – the difference is even larger.

 

I’m sure there’s an element of luck to that. And, historically, I hadn’t intentionally focused on low beta stocks. Though – on average – my stocks would have always had lower betas than the market.

 

I’ve run a portfolio for over a decade with 5 stocks that had less volatility than a portfolio (the S&P 500) with 500 stocks. So, whatever the theoretical math is – I can tell you that in real world performance having 1/100th the number of positions if they have 8/10ths or 2/3rds or half the beta or whatever may end up being no more volatile than the overall market.

 

If you go from 5 to 500 positions – you will basically go to a beta of 1 (it will be 1 if the 500 are the S&P 500). If you are getting 50%+ of the diversification benefit from just 5 positions – I’m not sure how likely you can improve on reducing the volatility of your portfolio by increasing the number of stocks you hold (because you’d have to find other low beta stocks to keep the beta of your individual stocks lower than the market).

 

I don’t pay attention much to diversification, beta, etc. But, if you are looking to reduce volatility – especially downside volatility – without reducing returns (and especially without lowering the chance you outperform the market in the long-run) all the research I’ve seen would suggest that holding fewer positions with lower betas would do more for an investor than holding more positions with betas closer to 1. Of course, holding a lot of stocks all with low beta would reduce volatility the most.

 

So, I don’t think diversifying into more stocks with higher betas actually reduces volatility as much as individual investors expect it too. These investors overestimate the importance of diversification in reducing volatility and underestimate the importance of simply owing low volatility (low beta) stocks.

 

Does that mean there’s no point to diversifying?

 

Actually, I think most investors will feel much better diversifying. This is not because their portfolio with more stocks will be a lot less volatile. It’s because diversification has 4 main psychological benefits:

 

1) It may make some investors more likely to just let some positions sit there. If an investor has only 3, 4, or 5 positions – he may sell one whenever he gets bored. But, if like Ben Graham did, he has 30-100 positions, then he may just hold on to whatever net-nets, cheap stocks, etc. are in his portfolio and remain cheap but are seeing no price movement. A big portfolio gives you lots of choices of what to sell and doesn’t focus you too much on whether each stock you own is “dead money” or not. So, that might be a benefit. Graham had pretty low turnover. Some concentrated investors have higher turnover. I usually do (definitely higher than Graham though lower than many modern investors).

2) It removes tracking error. Basically, if you hold 5 stocks – there will be quarters where you are down 5% while the market is up 15%. This looks terrible even if you outperform over time. A lot of investors want to avoid this. Their goal is to consistently eke out a small advantage versus the S&P 500. They want a “high batting average” instead of a few really big quarterly wins. This may be more likely with a slightly skewed but really big portfolio. I’m not sure. In the long-run, I’m pretty sure that if we’re measuring over 5 year intervals and not quarterly intervals the odds of the very concentrated portfolio outperforming are a lot higher. There are several reasons why this is. One is just statistical, a more concentrated portfolio should theoretically have better odds of both big outperformance and big underperformance. In practice, it seems actual concentrated portfolios tend to outperform rather than tend to underperform. That could be a research error like survivorship bias or something. Or it could be that only highly confident investors bother concentrating so the results don’t reflect what would happen if all investors concentrated. There’s another math issue here. Most stocks don’t just underperform the market. Probably – over their entire history – most public companies perform quite a bit worse than bonds. The constant outperformance of “the market” over bonds is due to a small number of stocks. Probably something like 2/3rds of all stocks don’t add – and may detract – from investment results versus things like bonds. All the positive results come from the 1 out of 3 (or less) stocks that manage to compound well long-term because they are good businesses. A concentrated portfolio may have a tendency to buy and hold more good businesses. This makes some sense. I know quite a few highly concentrated investors who are Warren Buffett type investors. I don’t know highly concentrated investors who are Ben Graham type investors.

3) It makes you feel less stupid. If you diversify widely, you won’t feel big mistakes are your fault. You can blame them on trade wars or interest rates or overall market trends like outperformance of growth over value lately. If I own only 5 stocks, then a bad performance has to be described as 100% my fault when writing to clients. For example, I lost 12% of my net worth in Weight Watchers (and more like 20%+ on paper at one point). I’ve talked about this mistake many times. It was entirely my own. Most investors can’t possibly lose 10-25% of their portfolio through decisions that are entirely their own fault, because they are so diversified that most of their returns come from market wide phenomena. Or at least “factor” wide phenomena like all low beta stocks or all low P/B stocks or whatever outperforming or underperforming over some period. It’s worth noting that this can be more psychological than really an actual impact on overall results. For example, that Weight Watchers mistake happened during the period where my portfolio had lower volatility and higher returns than the market while Weight Watchers also – for a little while there – was 25% of my portfolio. So, it can cause huge underperformance in absolute terms and especially versus the market for a few quarters or a year. But, the other 4 stocks are often enough to mute or offset this. Even with as small a portfolio as 5 stocks, it’s not unusual to have one stock flat and one stock up 50% in a single one-year period. You still get a big dispersion of results in a small set of stocks. If we just randomly picked 1/100th of the S&P 500, you’d still get one or two stocks out of those 5 with really unusual looking returns compared to the market.

4) You don’t feel like you are missing out on anything. In a diversified portfolio, you can always afford to own Amazon – just not a lot of it. My portfolio is so concentrated – about 5 stocks – that I say “no” to most stocks I actually like a lot. There’s just not a slot for that new stock. So, I often write-up a stock saying nice things about it, tell people it’s a stock they’d probably be happy owning long-term, etc. and yet I pass on it for my own portfolio. Most investors are not going to want to do that. So, most investors aren’t going to want to concentrate.

 

The experience of running a super concentrated portfolio is going to differ depending on the kinds of stocks in the portfolio. At Focused Compounding, we “focus” on just overlooked stocks. These all – by definition – have low betas. I say “by definition”, because it’s actually very hard to find stocks with a low ratio of average daily volume / total shares outstanding that simultaneously have a high beta. The reverse is also true. If a stock’s shares “churn” very rapidly, it’s hard for that stock to simultaneously have a low beta. As a rule of thumb, you’ll find that the more a stock’s shares turn over – the higher that stock’s beta.

Because Andrew and I focus on overlooked stocks – we end up with a portfolio with less volatility than the market, not more. We’re both big fans of Joel Greenblatt’s “You Can Be a Stock Market Genius”. And I’m sure there’s no way our returns at Focused Compounding will ever be as high as the amazing record he had while running his fund focused on special situations. However, it’s very likely that our returns – though produced by a portfolio every bit as concentrated as Greenblatt’s portfolio was – will have less volatility. Although we are big admirers of Greenblatt’s approach – we own stocks that aren’t really of the same “kind” as Greenblatt. For example, we own 5 stocks right now. I know that 3 of those 5 stocks have paid dividends for at least 50 years in a row. That’s more the kind of stock Buffett would own than Greenblatt would own. And it makes a meaningful difference in terms of beta. I don’t go out looking for “low beta” stocks. But, I do look for stocks where two things are true: 1) The stock is “overlooked” – that is, it has very low share turnover each year and 2) The company has been consistently profitable for a very, very long time. In cases where you get #1 and #2 – you almost always get a low beta. Greenblatt was looking at stocks that were sometimes actively disliked, confusing situations, highly leveraged, etc. rather than just being “overlooked”. If you read his book, you’ll see some of the cases he discusses were pretty well covered in the news. A lot of cases involved quite a bit of debt. And all of them – by definition, he was running a “special situations” fund – involved a lot of change. Stocks in the news, with a lot of debt, and undergoing big changes tend to have high betas – not low betas.

Most investors I talk to don’t appreciate the magnitude of the difference this makes. They want to know “how volatile is a 5 stock portfolio?”. There’s no answer to that. Our 5-stocks portfolio is not very volatile. Greenblatt’s was very volatile. If Buffett had a 5-stock portfolio – and, honestly, Berkshire’s biggest holdings picked by Buffett alone come pretty close to being just a 5 stock portfolio – it’d be somewhere in between. Buffett’s stocks aren’t “overlooked”. They are far too big to have low share turnover. But, they have very, very long histories of profitability. There are a few exceptions. Apple is the most notable. However, Apple is still considered a high quality and predictable company as of today. It just doesn’t have much of a stable record going back past the last 10 to 15 years. But, we can assume – if Buffett’s judgement of the business is right – that Apple is really in the same class of predictable businesses as things that have paid dividends for half a century.

So: how volatile is a 5 stock portfolio?

The answer is that it depends on who’s picking it. Our 5-stock portfolio would be less volatile than Buffett’s 5-stock portfolio. And Buffett’s 5-stock portfolio would be less volatile than Greenblatt’s. On average, our approach might be less volatile than the S&P’s 500 stock portfolio. But, on average, Greenblatt’s approach might be more volatile.

The reason for the difference is the beta of the individual stocks. If you owned 5 stocks with betas between 0.1 and 0.5 – your portfolio isn’t going to be anywhere near as volatile as you imagine a 5 stock portfolio will be.

On days like the recent Yuan devaluation day, most of the really “overlooked” stocks that have super low share turnover rates don’t actually move with the market. They just don’t. But, we own one NYSE stock. It moved every bit as much as the S&P 500 did that day. And that’s to be expected. Probably everything on the NYSE was moving a lot.

Any research you see about 5-stock portfolios is using something like randomly generated portfolios. But, in my experience, no investor who concentrates in as few as 5 stocks ever picks a portfolio that looks remotely random. I’ve seen 30 stock portfolios picked by active managers that look random. I’ve never seen a 5-stock portfolio look random. The reason for this is probably the extreme concern the manager has with the large amount of risk each stock could pose to the portfolio. If a stock you own goes to zero – you lose 20% of your net worth. A lot of “random” stocks go to zero. Therefore, a manager who holds just 5 stocks at a time won’t feel he can afford randomness in his portfolio. So, I think experiences like Greenblatt’s and like my experience are a better indicator of what volatility to expect in a 5-stock portfolio than randomly generated 5-stock portfolios.

The answer, then, is that the volatility in a 5-stock portfolio will depend a lot more on the betas of the individual stocks in that portfolio than most people think. One reason for this is that the influence of diversification on volatility between a 5 stock and 500 stock portfolio isn’t as big as most investors think.

I choose to have a 5-stock portfolio instead of a 25-stock portfolio. However, I wouldn’t recommend that to most investors. I would, however, recommend to everyone that a 25-stock portfolio makes more sense than a 500-stock portfolio simply because going from 25 to 500 stocks would put an incredible strain on your stock picking efforts and yet wouldn’t make a big difference in volatility. If you’re ever tempted to buy a 26th stock, just get out of active investing entirely and put everything in the S&P 500.

I think my approach of owning 5 stocks that are each lower volatility stocks than the market might be psychologically workable for some investors. Maybe it’s 1 out of 20 value investors or something. Greenblatt’s approach – of focusing on 5 or so high volatility, special situation stocks – is likely to generate the highest returns. But, could more than 1 in 100 value investors stick with such an approach? I don’t know. A lot of people are uncomfortable with my approach – and it’s not even more volatile than the S&P 500.

Why is that? I think it’s due to more investors being educated about the benefits of diversification without being educated about the benefits of low beta. Everybody has been told to diversify. Not as many people have been told to focus on less volatile stocks. But, I think both diversification and owning less volatile stocks are helpful tools in getting investors to the level of risk they’re comfortable enough with to stay in stocks 100% of the time. And that should always be the goal. Long-term results for investors will be best in whatever strategy keeps them 100% invested in stocks 100% of the time. The biggest risk to long-term results is usually an investor’s own tendency to sometimes be less invested in stocks than they could be. And that kills compounding.

Even for randomly generated portfolios the difference between 25 stocks and 500 stocks can’t be meaningful enough to ever mean that investors should go over 25 stocks with diversification as their purpose (maybe, like Graham, they just want to own very small stocks in a not as small fund – then, they’d end up with positions that couldn’t add up to like 4%+ of their portfolio).

 

There’s a huge difference in diversification between owning 1 stock and 5 stocks. And there’s some difference in diversification between 5 and 25. But, there’s not a meaningful benefit to anything beyond 25.

 

The thing you’ll see value investors quote the most is Joel Greenblatt – early in “You Can Be a Stock Market Genius” – who broke down the amount of diversification’s benefit for each number of stocks.

 

But, I want to stress something here. The reason why so few stocks can diversify away so much of the “specific” (that is, non-market) stock risk is that the market volatility is so huge. Basically, if you have a stock portfolio you have a 2 out of 3 chance – in any year – of having returns between -10% and positive 30%. That’s a lot of variation. And that range is going to be pretty close to the same whether you own 5 or 500 stocks. It’s not really that 5 stocks are all that diversified. It’s that even 500 stocks have such a huge range of outcomes.

 

I’m skeptical of all of this data though – because, we’re really talking about 1-year type returns. If you bought a portfolio of 1, 5, 25, or 500 stocks today and held for 10, 20, or 30 years before selling anything – you might get a very different results. The risk of buying AND HOLDING 1-4 stocks (for example) would be much, much higher than what this data shows because of the way compounding works. It’s entirely possible to construct a portfolio of very few stocks with no long-term winners in it. It’s nearly impossible with 500 stocks. If you don’t sell, the long-term winners became a bigger and bigger part of your portfolio. Those winners are what really drives market returns. Most public companies are bad investments over the long-term. But, most indexes continue to hold long-term winners for the long-term at the same (or actually, even higher) weightings as those businesses compound their value.

 

So, I’d be very skeptical of applying any of these ideas about diversification to buying and holding groups of stocks. Obviously, you are more likely to have a few long-term winners in a 25-stock portfolio than a 5-stock portfolio. This may help protect against underperforming bonds with a buy and hold strategy.

 

If you turn your portfolio over a lot, then both misjudgments and correct judgments of long-term business quality will matter less. The best chance for really big underperformance would be buying a small number of stocks (1, 3, 5, etc.) and then holding them forever no matter how badly they performed. This would give you the biggest difference between your portfolio and the market. If you were a great business analyst – you’d get a great result. If you were a bad business analyst – you’d get a bad result.

 

Volatility math is pretty complicated in that I don’t think most studies are measuring what most investors are interested in. Most investors are probably interested in “downside volatility” only and in the chance their portfolio will fail to compound at a rate above bonds, closer to the historical average in the S&P 500 index, etc. over the long-term. I’ve read very few studies where that is what academics are actually trying to measure.

 

From what I have read, I would suggest this. One: beta is generally bad. It’s associated with both higher volatility and lower returns. There’s nothing good about it empirically. And just looking at it theoretically too – we should be skeptical that high volatility stocks will lead to good long-term returns. The S&P 500 isn’t going to go to zero. But, any one stock might. If – for a second – we pretend a stock is not part ownership in a business, but instead just a piece of paper that wiggles around in price, then we’d DEFINITELY want to avoid anything with high volatility.

 

Why?

 

The volatility you are looking at is for a small slice of time. Let’s use a year as an example (though we could use a day or month). Pretend the market has a one-in-two chance of being down 10% for the year or up 30% for the year. It doesn’t. That’s understating its actual volatility in the past. But, pretending that makes this an easier calculation. It might seem like you’d really like a stock that was just like the market but 3 times more volatile – that is, if you flip a coin and it comes up heads you are down 30% (instead of 10%) and up 90% (instead of up 30%). But, you aren’t going to play one round of this game. You’re going to play the game for life. So, you’ll be flipping that coin every year. If you plan to play this game for 30-60 years, the chances you will get 5 heads in a row aren’t low (I think you’d expect about 1 series of 5 bad flips in a row every generation). That means you’d see about an 83% decline (1*0.7*0.7*0.7*0.7*0.7=17%) quite often. And, remember, I really understated the base volatility of the market. But, in my example the market would be down like 40% after 5 bad flips and something 3x as volatile year-to-year would be down closer to 85% after the same 5 bad flips.

 

A random walk is going to result in total ruin a lot in volatile stocks. A lot of those kind of stocks are eventually going to end up being worthless.

 

Now, this isn’t the way I think of stocks or of risk. I don’t think of risk as being the volatility of the stock. I look at it from the long-term survivability of the business. That’s what makes a stock safe. But, if we pretend that risk is the volatility of a stock – it’s very important not to confuse what seems to be a manageable level of volatility in one snapshot of a film (a single year) with what is manageable if you run the film for 30 or 60 frames (an entire investment lifetime). Because of the way compounding works, it’s not surprising that if you were to compound both upside and downside risk at higher than normal levels – you’d often randomly walk yourself to zero over a long enough series of years.

 

Because investors tend not to own stocks long enough and academics tend to measure very short time periods – the risk of owning stocks that are so risky they destroy value over time isn’t obvious to us. But, it’s obvious in the record of indexes. Indexes are the one place where stocks are simply held for very long periods of time and academics look at the long-term results. The result of an index versus the actual result of a “median” stock selected in that index is very different. I don’t need a research paper to tell me that. I’ve run plenty of back tests of net-nets and predictable stocks and so on over even periods as short as 10-15 years to see that you end up with fewer, bigger winners and lots of really bad losers than you’d expect. In the long-run, the relationship between a stock we pick today and hold versus an index is quite weak. It’s a lot weaker than the relationship day-to-day.

 

On any given day, most of the stocks I own are going to have moves that look a lot like the market. Over a decade, they won’t. It’s very important to think about what time periods we’re measuring over. Investors underestimate how much of a stock’s move over the last 30 days has to do with the market and they overestimate how close the stock’s move over the next 30 years is likely to be to the market. Very, very few stocks will end up having returns anywhere near the CAGR of the market over the next 30 years. What seems like a real small difference this month – in terms of returns, volatility, etc. – when compounded over 360 months would become a huge difference.

 

That’s one of the 2 reasons I think it’s not a good idea to put much faith in most of the academic stuff you’ve read about ideal portfolio size and diversification.

 

1) The portfolio is assumed to be randomly selected (in reality, the smaller the portfolio – the less random it’s likely to be due to human biases that’ll be a lot stronger the smaller the portfolio they are building is)

2) Volatility is almost always measured over too short a period of time

 

For this reason, I think most things you read would lead you to underestimate just how long-term risky a highly volatile, super concentrated portfolio is and to underestimate how safe a low volatility, super concentrated portfolio can be.

 

The stuff Warren Buffett owns is just not remotely comparable to a randomly selected stock. Buffett’s outperformance may be due to skill that other investors can’t replicate. But, other investors aren’t even trying to replicate the safety of what he’s doing. So, if your average investor tried to concentrate like Buffett – the result could be much, much uglier than even models suggest. I’m talking in the long-run. That’s because the average investor isn’t just going to pick just all around generally worse stocks than Buffett. I can tell you right now the main difference between what they’d pick in their 5 stock portfolio and what Buffett would. The average investor will pick more stocks that end up going to zero. They’ll pick risky stocks – worse businesses. That’s the huge difference I’m sure of.

 

Randomly generated portfolios are a little weird. They may make sense in modeling the behavior of some investors. But, they include a lot of stocks that are just much more risky long-term – things that seem like, if given enough time, they will eventually go bust – that someone like Buffett would never pick.

 

So, when looking at the number of stocks in a portfolio – I think it’s important to consider what “kind” of stocks they are and to think long-term. The month-to-month volatility isn’t what matters. It’s the long-term risk from what you own. So, having 5 stocks might not be very risky for me. Certainly, dropping from 5 stocks to 4 stocks (by just adding to the other 4 stocks) wouldn’t make much of a difference to volatility in my portfolio. But, then, adding a stock like Weight Watchers to go from 4 to 5 stocks would make a big difference in volatility, risk, etc. Because Weight Watchers was a different “kind” of stock from most of those I owned.

 

As far as trying to figure out the exact math on the right number of stocks to own for diversification purposes – I think that’s silly. One stock is not enough. A lot of the benefits of diversification are reached by 5 stocks. And there’s no point diversifying beyond 25 stocks. I think that’s all an investor needs to know. The question then becomes what “kind” of stocks to fill those portfolio slots with – not worrying about how many slots there should be.

 

Share: