I think its helpful to realize that the S&P 500 is not some static index of companies whose selection has been on auto-pilot but actually a fairly actively managed index whose selection algorithm is continuously being updated and refined. There is a not insignificant human judgement component of how to define the selection/weighting criteria.
In some senses, behind the scenes its likely similar to the Twitter/FB/Netflix algorithm - at some point there is some editorial opinion being inserted via criteria and weighing of those criteria.
"Some years ago Bill Miller, then a well-known Legg Mason portfolio manager with a stellar record of beating the market, noted that the S&P 500’s track record of being very hard to beat suggested that active management can succeed and that the Index Committee were actually good active managers."
Yes there's an index committee comprised of humans. But the index methodology is public and when there are tie break decisions needed, they're fairly predictable.
Another thing to think about is that most of the outperformance of the index is driven by the largest companies. But not because they were added when they were their current size. But because of the buy and hold approach of getting them in when they're still on the small end of the large caps. There was a good recent paper on this concept, showcasing that most of the big index returns are attributable to only a few stocks. Similar to how most of global market returns are mostly just US.
Is the algorithm "select * from companies order by market_cap desc limit 500"? At the risk of oversimplifying, there should be quite little to debate, and when some hair-splitting occurs, I trust a committee of reasonable humans will make a reasonable choice.
The algorithm is a little more complicated than that, as various judgements have been added in, but always with specific rules. When there was a trend of tech companies going public with dual classes of shares leaving founders in control forever (e.g. Zuck with Facebook) they announced that any company which went public after date X with dual classes to prevent the economic owners from exercising control would not be allowed onto the index. Similarly, there are rules about profitability that, IIRC, were added after the first dotcom bubble to try and keep extreme bubbles from mucking up the index (I think it's that you need to have two consecutive quarters of profitability, but I don't remember the details).
Also, the market cap of company 450 and company 550 are close enough that recent market performance can flop ]back and forth fairly regularly, which the S&P has further rules to try and limit, though again I don't remember the details. Again, they try to be fairly rules based, but do exercise some discretion in creating the rules.
You have to adjust for free float market cap. Companies need to be US incorporated. You have to be aware of share classes. For example both GOOG share classes are in the S&P.
When it comes to tiebreak decisions, there may be 5 or so potential adds when there is 1 delete. There may be a focus on evening out sector exposures, preferencing underweight sector adds. (Just my guess)
That's interesting. I was contacted by a personal money manager looking for clients and he explained his philosophy as sampling 30-50 stocks from the top end of the S&P 500. He had outperformed the market during a bull run, which I attributed to the increased risk of a smaller portfolio. I didn't invest because I figured his fund would underperform the index during the next bear run for that same reason, canceling out the gains and moreso with fees. Maybe there actually was something to his strategy; I haven't followed up on his performance.
Owning a handful of shares does capture a significant amount of the beta for a market.
However with very significant profitability outliers, such as power law distributions, then you have to own all the shares to ensure you don’t miss out on the one or two shares that make up most of the gains. Venture capitalists talk about this a lot.
Historically the same issue occurred with market timing: most market gains are made during only a few relatively short unpredictable periods. You need to own shares for a long time to ensure you are in the market when those few unpredictable ups occur.
IIRC many active managers beat the market/S&P 500/benchmark before fees. It's almost impossible to beat after fees because the fees are too much of a drag.
I think the point is that the divergence in market cap is so large that it’s become equivalent to a highly opinionated active manager. Any fund that’s allocating over 13% to just two companies seems a bit risky, but that’s what you’re getting with Apple and Microsoft here.
I don’t think it’s as opinionated nor active as you’re making it out to be.
Investing proportionally more in a huge company than you do in the 500th largest eligible company seems much more passive than active to me.
If two S&P companies were to merge and everything else remained the same, I’d think that having the combined amount invested in the combined company to be more sensible than cutting your investment proportion in half merely by a merger.
Or if a company you’d invested in doubled in value, I’d rather hold the constant number of shares rather than sell half of them to bring my exposure to the winner down. I’d also view that as passive and un-opinionated. Selling half feels more active and opinionated than simply holding.
I’m sorry, but this is just not true. Active managers are measured by their deviation from the S&P (i.e. a beta 1 manager is extremely passive). This index has also always been concentrated to some extent, so not sure it’s any kind of recent change.
I think maybe you are talking about risk, but it’s a completely separate idea.
I was confused by NVIDIA having a similar weighting to Alphabet despite having less than half the market cap. But notice that Alphabet appears twice. (GOOG and GOOGL.)
The SP500 index might be a little more complicated than just the 500 biggest public companies. When Tesla was rejected despite meeting criteria, it was odd. However “edition” impacts less than 1% the actual value.
> * There is a not insignificant human judgement component of how to define the selection/weighting criteria.*
Not wrong, but the rules are relatively fixed and known ahead of time and somewhat more deterministic compared to the decision-making process of most active funds.
It should also be noted that the S&P 500 isn't the only index. The Russell 3000 and Wilshire 5000 try to cover "all" publicly trade companies in the US:
> The terms passive investing and index investing are often intertwined, but they are not exactly the same thing. Today’s guest is Adriana Robertson, the Honourable Justice Frank Iacobucci Chair in Capital Markets Regulation […].Adriana is interested in index investing and, in this episode, we hear her views on whether or not index investing is passive. Hear facts from her paper on the S&P 500 Index fund specifically, and all of the reasons that it's not passive, as well as some of the issues that are potentially arising from the creation of so many indexes or so-called passive investments. A more recent paper by Adriana, published in The Journal of Finance, surveyed a representative sample of U.S. individual investors about how well leading academic theories describe their financial beliefs and decisions, and Adriana shares the differences in something like value growth from an academic perspective versus a real-world perspective. Find out how investors can go about evaluating the performance of their portfolios and what they should be looking for when deciding which index fund to invest in, as well as why index funds aren’t a meaningful category anyway, factors from Adriana’s surveys that might influence investor’s equity allocation, and the trend towards indexing and whether it will overtake active portfolios. Tune in today for all this and more!
Agreed. I have ruined so many dinner conversations over this. You can't beat the SP500, it is the only free lunch. It is truly depressing when you think about it - you just feed the machine. The American millionare class is a group of 401k holders. [Oh. and they made 54.96% in the last 5 years]
Your best odds of being an american millionare is pushing all you can into a 401k which purchases sp500. Most american millionares are not business ninjas. They are teachers and firemen pushing into their 401k, they also don't make much money. "Only 31% averaged $100,000 a year over the course of their career."
What happens is the ridiculous return of the sp500.
Yes but the vast bulk of wealth is owned by a very small minority. That's who people are generally talking about when referring to the "millionaire class".
Forward looking statements made with such confidence tend to be wrong. I take it you will personally make whole anybody who follows your investment advice in case you are wrong?
They are not actually selling stock that's not performing well and buying stock that is, though, are they?
It's very hard to beat their performance if they can pick winners at any time and just discard losers without any actual loss.
This is going to change because too much money in indexes means not enough capital is chasing winners, and is instead enjoying the rising tide. That is okay and it is sustainable. But it still means that the upside for the ones who do chase winners and who succeed in placing correct bets on great companies will be more and more massive. Because the delta in capital from the index to the cream of the crop will be way larger in absolute terms as the indexes grow in size, and the winners will continue to prop up a huge rising tide of continued innovation.
I’d be worried if nobody could beat the market. 7% to 15% beating the market seems reasonable to me. The market is an incredible abstraction and it works, and it’s also still great and reassuring to know that there’s still an echelon of actively managed funds that beat the market.
Agree, ~7% seems ok? The funny thing about the profession of active fund management is that there is a ‘scoreboard’. Almost no other profession can or would score itself like this. If they did, I doubt more than 7% would beat a suitable benchmark for hairdressers, plumbers, doctors etc.
This ‘score’ has other useful properties. A successful fund manager has to operate under difficult and changeable conditions for long periods of time. They can’t get there just by default or by being lucky. Knowing which people can do this very challenging task means the advice they give about investing, managing your own psychology and life in general is imbued with a legitimacy that is difficult to find elsewhere.
>They can’t get there just by default or by being lucky.
Why do you say that? If I ran a League of Coinflip Guessers with 10,000 entrants, someone would still win the league, and probably with an astonishing score too. Why not the same with stock returns? You have 10k managers it stands to reason some of them will by pure chance beat whatever metric you set.
I guess the analogy would be if every year the same people keep finishing in the top quartile in your coin league, at the same time as failing to make the expected losses dictated by chance.
That's a very good insight. The success of the index funds is nothing but an indicator of the inflation from the top, that has not trickled down to Main Street until recently. In a stable solid monetary regime, capital is forced to pick individual stocks, as opposed to riding the inflation wave.
If you read the original bet and discussion carefully, what Buffett really bet against was the fees. It’s a very important detail in the original bet that the returns would be compared net of fees. His argument was not that active management was doomed to fail, but rather that active management is usually not worth what you pay hedge fund managers to do it.
Well he hasn't exactly bet against active management since all his wealth is actively managed. A more accurate interpretation would be "what I'm doing is incredibly difficult and I bet most who try will fail."
Given the sheer size of “mega cap stocks”—being Apple (AAPL), Microsoft (MSFT), Amazon (AMZN), Alphabet (GOOG & GOOGL), Meta Platforms (FB), Tesla (TSLA), NVIDIA (NVDA), and formerly Netflix (NFLX)—most portfolios carry noteworthy exposure to these companies.
When stripping out the entire cohort of Mega Cap Stocks, the S&P 500’s annualized performance in the five-year period ending February 28, 2022 would have fallen to 6.98% from 15.17% annually—a reduction of more than half. The index’s standard deviation would have seen an improvement of 4 percentage points, but the lost returns seem to sting more than the reduced volatility.
The pattern persists in each full calendar year since 2015. When each Mega Cap Stock is hypothetically excluded from the S&P 500’s total return, the index suffers. Notably, 2020’s 18.4% total return for the S&P 500 would have been a mere 3.3% had the eight Mega Cap Stocks been excluded, by our calculations.
Only in 2022 (year-to-date, through February 28, 2022) would removing the Mega Cap Stocks have actually boosted the S&P’s growth.
The point is that there's no way to know which "7" companies will carry the bulk of the growth, so you grab some representation of "all" of them.
So yeah, what you're saying is exactly how it's meant to go.
Also, "recession is looming" is easy to say at literally any time in history. Eventually, you'll be right. The trick is to know precisely when it will take place.
The idea that there are lots of places lying around to put 10's of billions to work at high returns, and the company just isn't being creative enough, is preposterous.
Stock buybacks were outlawed for decades as insider trading. At some point it became legal, people made money, and all sorts of verbiage piles on when people make money. It’s still insider trading and insider trading does things like mucking up actual market valuations and adds leverage for a later correction to unwind, dramatically.
Companies that issue dividends usually have that dividend as a prominent feature within their business case. As in “we send out 90% of our profit as dividends after all costs/maintenance are paid.” They’re also usually almost cookie cutter businesses like REITs or mines. (Cookie cutter as in there’s well established business models already in place for decades/centuries. There’s differences one company to another but it’s characteristics of the assets of that company which change not fundamental components like renting out owned assets or digging up rocks from the ground.)
Compare that with returns to technological innovation. Econ defines tech innovation as “growth in profit not attributable to any other traditional business operation.” In a sense, a “tech company” retiring cash to their market cap is saying they can’t think of any other thing to do with that cash, including growth of their bottom line ie technology. Quitters mindset.
The obviousness of it being insider trading probably becomes even more glaring if we could see the machinations of a stock buyback. They’re probably timing the market to make sure the price increase is most pronounced or meets some metric that’s dubious.
Further, it’d be interesting to see how the stock market would respond to a company class defining themselves as stock-buy backers ala “we use 90% of our profits to buy back stocks” like the more traditional dividends. Would the stock value reflect today all the expected stock buybacks in the future at a time value discount?
Stock buybacks are not dividends. Dividends are paid out to shareholders at a declared date and come from a defined source of funds. The separation of the source of funds for a dividend and the rest of the companies accounts is clear cut. Buybacks are murky in comparison. If the market or stock has a bad day on the day of executing the buyback the actual buyback could be negated by the overall trend. Further, a company would logically not buyback their stock on such a day unless it fit their criteria otherwise. Ie insider trading. It’s not possible to call dividends insider trading in the same sense because dividends are paying out shareholder income.
Companies doing contrived "creativity" can be worse for the company than reducing the number of outstanding shares with a buyback which when done appropriately is a healthy reorganization of the capital stack.
How many mergers and acquisitions have we seen epically blow up? How many billions down the drain on projects like the windows phone or Google+? Hindsight is 20-20 but when it comes to pouring tons of money into new products we're not exactly blind either.
Again, by what criteria are you saying we're currently in a recession? Every period in https://fred.stlouisfed.org/series/GDPC1 shows that we're not in a recession when our GDP is currently rising consecutively...
Your original comment was insinuating political figures are changing the definition. I'm assuming you mean if we just had 2 consecutive quarters of GDP decline then that definitively means we're in a recession... But that is objectively not the case by the figure you just told me to look up. We're in 2 quarters of real GDP increase.
A recession ends when the economy recovers, not when there's one quarter of GDP growth. By your logic you'd have to say that we've had two separate recessions in the past 3 years.
<5% of companies have driven most of the returns of equities:
> We study long-run shareholder outcomes for over 64,000 global common stocks during the January 1990 to December 2020 period. We document that the majority, 55.2% of U.S. stocks and 57.4% of non-U.S. stocks, underperform one-month U.S. Treasury bills in terms of compound returns over the full sample. Focusing on aggregate shareholder outcomes, we find that the top-performing 2.4% of firms account for all of the $US 75.7 trillion in net global stock market wealth creation from 1990 to December 2020. Outside the US, 1.41% of firms account for the $US 30.7 trillion in net wealth creation.
> Four out of every seven common stocks that have appeared in the CRSP database since 1926 have lifetime buy-and-hold returns less than one-month Treasuries. When stated in terms of lifetime dollar wealth creation, the best-performing four percent of listed companies explain the net gain for the entire U.S. stock market since 1926, as other stocks collectively matched Treasury bills. These results highlight the important role of positive skewness in the distribution of individual stock returns, attributable both to skewness in monthly returns and to the effects of compounding. The results help to explain why poorly-diversified active strategies most often underperform market averages.
The trick is knowning which company(ies) will be that 5% and when: some do well for a time and then fade away: you would have to know when to jump in and out of them.
Also, the S&P 500 being concentrated is not new and has been the case for 40+ years:
If you had invested in a fund that just tracked S&P 500, what would have been your, over, say, the last ten years, what would have been your returns? I can't figure out if this info is in the article.
Are there low-fee funds that just track the S&P 500? That would seem to be the way to go for long-term investing? What such funds can I find? (taking into account as other comments in this thread point out that, yes, the S&P is actually actively managed itself, sort of).
I'm continually astounded by techies who have so much money and so little idea what to do with it. It's not like this stuff is even hard, not compared to keeping up the latest JS nonsense: https://www.bogleheads.org/wiki/Main_Page#mp-gs-h2
You may be over-estimating how much money I have as a techie, I work in non-profit/academic sector. My retirement funds are relatively paltry compared to what you may assume about techies with so much money (but perhaps still more than US median for my age; googling says median US retirement savings across all working-age households is $95,776), and mostly in managed 401k/403b where I have previously just chosen "lifecycle funds". ¯\_(ツ)_/¯
People make a lot of assumptions about how much wealth random HN commenters or the HN audience in general have, and sometimes I start feeling like I'm really poor compared to all you techie multi-millionaires... but when I actually pay attention to comments, I realize, nope, a lot of HN is more like me (although probably often ashamed to say so).
If by "lifecycle funds" you mean target date funds, setting your allocation to 100% for your retirement year is the most reasonable strategy you can do.
But isn't that what the "lifecycle" target date funds do? The argument is that you will know the schedule you want to rebalance on for your custom personal needs, better than the generic target-date "lifecycle" fund will?
Sorry, I didn't check comments for a while. The target date funds tend to have higher management fees and yes, worse composition of stocks versus bonds compared to holding a mix yourself, IMO, based on your goals.
You're welcome! Sorry if that came across as aggressive. I get tired of seeing people on HN posting like, "How do I live on my paltry $500k/year income? Should I gamble it all on dogecoin?" (this is an exaggeration, but only slightly)
"...If you had invested $10,000 in the S&P 500 index in 1992 and held on with dividends reinvested, you'd now have more than $170,000. The market volatility in 2022 could cause this return to decline somewhat. However, the index has proven to be a winner over the long term..."
If you win a lump sum, studies show that the best time to put it in the market is all of it right now, not trying to time the market or DCAing it into the market. Of course, you could get unlucky, so that you're initially in the 2000->2013 style window, but you can't know that at the time.
Of course, most of us do / should invest smaller amounts over time, just because that is what our earning profile is like.
> Of course, you could get unlucky, so that you're initially in the 2000->2013 style window, but you can't know that at the time.
2000+ was only an issue for US-only, equity-only investors: if you were globally diversified you were fine. Even if you were US-only, but had at least 20% bonds, you were also fine:
Which is the whole point of DCA. It makes it less likely to get unlucky. (It also makes it less likely to get lucky. DCA makes it more likely that you will simply regress to the mean.)
To be fair, I think the claim is that the expected value of your return is maximized by dumping it all in today. However, I believe the variance of your return should go down as you spread out the investments, no?
Ben Felix's channel ia a great place to find empirically based investing advice. All of his videos cite the papers where the information he spouts can be sourced. About lump sum investing, you can check this video:
“Value small caps” is the consensus among Ben Felix viewers, but not really outside of that group as far as I know. He’s very convincing but I’m not sure I’d recommend everyone does that, compared with buy and hold S&P 500 or total market index.
You received one good set of links, but I'm pretty sure I first read of it from a Meb Faber study that I'm not in a position to look up right now. He has done a lot of work on simplifying and making investable decent and straight forward strategies for long term investment.
Or, if you had invested after 2008 and kept investing every year, you’d find yourself gaining money every year, and start to think “this shit is easy…always goes up”
OK thanks! now to figure out how to calculate that as an annual rate of return (APY?) Sounds like this was through 2021, already a bit old?
If I plug this into a random calculator on the web without knowing what I'm doing, $10K to $170K over 20 years looks like... around 15.25%? That does seem quite high, can that be right?
Thanks, that seems more realistic than the 15% I incorrectly came up with. 10% is what I had in my head as a very good but possible managed fund return -- and only 8% over the past 20 years instead of an unrealistic 70. This seems like a more realistic expectation.
So a conclusion might be: if you've been getting less than 8% (after fees) over ~10 years in any managed fund... you might want to reconsider your investments, seems like. But if you've been getting 8-10% or more, you're pretty good. Does that make sense?
I hope you realize these are extraordinary returns. Show me an active manager that can demonstrate consistent 8% returns over the last 20 years, and I will be at their door next Monday 08.00 AM.
This page from Jan 2022 suggests average fund returns over the previous 15 years (so that did include the bear market 2008, at that point, the article mentions) for "U.S. Large-Cap Stock" was 9.73%
It does note that that includes "the average for all mutual funds, including index funds." But if the answer is "oh yeah, the index funds bring up the average a lot"... why would anyone use anything but an index fund?
Is it going to be a lot lower than that over 20, even though that already includes 2008?
I admit this is all pretty confusing to me.
What returns would you consider acceptable over 10-20 years? Or do you not even look at past returns when deciding whether to keep your money in a given fund, or invest in a given fund?
I am with Nicholas Taleb. Past returns are no guarantee. And in any case...
“They will envy you for your success, your wealth, for your intelligence, for your looks, for your status - but rarely for your wisdom.”
― Nassim Nicholas Taleb, The Bed of Procrustes: Philosophical and Practical Aphorisms
I mean, that makes sense, but I'm having trouble understanding the practical implications.
If not on past returns, how do you decide where to invest your money? Have you ever decided to remove your money from a fund, on what basis would you make that decision?
There is _no_ returns that would cause you to re-evaluate your current investments? If you were invested in a fund that had given you say an average of 0.5% a year over the past 10 years, that would not cause you to re-evaluate your investment?
> If you were invested in a fund that had given you say an average of 0.5% a year over the past 10 years, that would not cause you to re-evaluate your investment?
What about dividing funds for investment in three parts?
In a basic theoretical model excluding things like individual holders, short term speculators and stocks outside of the index, if you add up all the active funds, then the holdings should indeed match the passive funds (since the passive funds are just holding a percentage of the total, the remainder must hold the remaining percentage).
So in this toy scenario obviously active funds would underperform after fees. But there's more reason for investors to pick stocks than just beat the market (risk preferences - for a lot of people the benchmark is not the S&P but bond yields). The majority of active management is not hedge funds and are not in the business of providing outsized returns (especially not to small retail clients). To beat the market, one must take risks that the other active managers or index committees deem to be unacceptable.
The issue is that some active funds do beat the market... And they do consistently. But the buyin is high and the fees are above industry averages. So people look for lower cost active returns and fail.
But normally they do it with closed money pools. In the meanwhile they normally have other money pools that offer to their customers...but the algorithms never seem to work as well for those money pools...
I have an economic question. Several people in the hedge and equity industries have asked that hedge funds in general pay a 5% tax. If that were to happen how would that change the outcome as in such a situation one has to factor in making enough on investments to cover the 5% tax?
Or in general what can be done to economically align hedge funds with making some basic 5% or more return?
The SEC is regulating public companies too much. This leads to higher compliance costs, which decrease shareholder returns, and to more and more companies opting to remain privately held rather than going public, which is siphoning sophisticated investors away from the stock market and toward private equity investing.
It's hard to beat index fund because everyone is buying and holding. 401K is now total like $7-8 trillion dollars. Soon, baby boomers will start the withdrawal wave, I don't know if index funds will be as hard to beat then. I hope so because my retirement is riding on 401K.
In the olden days when most companies paid dividends, retired people would try to keep owning the shares and live off the dividends.
The actual amounts sold by retirees are probably low (say a few percent a year), although I would like to see some numbers per age group. Presumably it is common for sudden death and their shares to be passed on to children?
Sort of a side note, but living off the dividends is mostly a psychological thing. The stock price goes down by roughly the dividend amount around the ex div date since a company with less cash on-hand is worth less. If you own shares of company that looks like it should pay a dividend but doesn't, you can simulate it by selling shares.
That said, dividends are a decent proxy for company stability and health, but execs also know this, so they play games with dividends to keep income investors interested.
The older group of gen x will be withdrawing soon. Also, more participation doesn't mean more money, at least from what I heard last time. Meaning boomers, gen x contributed more due to their career track vs millenials & gen z.
Part of me feels like the market is a pyramid at this point, newcomers are paying for older generation in hopes the next generation will do the same.
