Edited to add epistemic status and summary:
The basic thesis of this post is that, for many reasons, the volatility (riskiness) of buying a house are significantly higher than generally believed.
The considerations explored here are the result of discussions with a financial advisor and plugging various pieces of publicly available data about housing markets into adhoc models of long run performance. Ultimately, predictions about asset classes usually come down to reference class forecasting, as any appeal to 'over' or 'under' priced is relative to some trend, and choice of the timescale on which trend following is expected can be chosen to tell a story. E.g. two people arguing can have one arguing for a reversion to a long run historical trend while the other argues for a reversion to a short run trend. They may both be right on different time scales going forward too. This post attempts to surface reasons why buying may be more expensive than a popular model one might encounter, which may have been generated by people in the pro real estate camp. Thus it argues that people's confidence intervals, especially to the downside, are too narrow when modeling home buying. Losses in the housing market are hard to get quantified data on since they are private and often vary enormously due to imputed rents and opportunity cost being idiosyncratic between individuals. A small thumb on the sale at each step in a multiplicative planning space can add up to a lot by the time you reach the output of your model.
The main parameters looked at in this post if you want to do you your own homework (which obviously you should for an individual purchasing decision) are as follows:
Rent-to-Buy ratio (see linked post at bottom for problems with the default parameters of popular rent to buy calculators)
Average time spent in purchased homes
Historical house price indexes
Historical rent price indexes
Vanguard's Balanced Index investment fund (leveraging a balanced fund handily beats all-stock allocations btw)
Salary and savings considerations over a career
Home ownership tax benefit calculators
I'll add any more that occur to me.
Beware extrapolating averages to unusual situations. Outlier markets usually have various reasons that have driven metrics to extremes. Failure to understand the casual mechanisms will prevent one from predicting when such trends might reverse.
The average person spends about 10 years (87k hours) in a house before moving, which already goes against people's tacit models. This post is about the various reasons why buying a house is a bad idea.
First let's address the idea that renting is 'throwing money away'. This ignores the opportunity cost of investing extra income and the lump sum of the down payment into a house instead of the stock market. People spend on average 50% more money on mortgages, taxes, and fees than they spend on rent. If that money were invested in the market it would be earning you a return. People very often report being surprised how much more a home wound up costing them than they had expected. You can plug in the numbers for yourself in a rent to buy calculator (and should do so do get a sense of which numbers matter, but keep in mind they tend to have defaults set that are crazy and home buying tilted). A financial planner goes into more details about rent to buy ratios here(link below). Summary: buying makes sense in growing markets where the rent to buy ratio is under 20. But the markets in which the rent to buy ratio is under 20 are the same markets that you *don't* want to live in. The ratios are low precisely for the reasons you don't want to live there, various quality of life and job prospects.
Second let's address the fear that 'rents keep going up so it becomes a better deal over time.' Rents can't go up relative to house prices or vice versa for very long because eventually people will arbitrage the difference away. When renting is cheap, demand for purchasing homes drops. When renting is expensive demand for purchasing homes goes up. But we'd expect small persistent distortions in the direction of buying being more expensive because so many people default to buying even when it doesn't make sense. But what about the Bay Area? Compound annual growth in housing prices over the last couple decades has been around 7.5% while growth for rent has been around 4.6%. But this is exactly why I expect regression to the mean. Overall rent to buy ratios in the Bay have been overheated and are now dropping, and it's prices that are getting pulled down faster than rent is going up. Additionally, rents have historically appreciated much more slowly than the stock market, meaning renting should get cheaper for you over time if you're investing.
Isn't that all moot if the home you buy keeps appreciating in price since you get to buy with leverage? (A down payment is typically 20% so a home purchase is leveraged 5:1). In the fastest appreciating housing markets, this has been true. But buying a single house is like buying a single stock. The volatility you should expect is going to be much higher than a diversified stock portfolio. Essentially, any claims made about predicting future housing returns boil down to momentum based reasoning vs regression to the mean based reasoning. Either could turn out to be true, and you shouldn't assume that you're going to out guess the market. And with all those wins you hear about you have to account for survivorship bias. All the people who didn't make out so great aren't bragging about it.
Take housing the in the Bay Area for instance. Apparently more than half of home sales are going to Chinese buyers. If there is a substantial crack down on the various schemes these buyers are using to get money out of China, or if a different city becomes a hotter investment market, or unforeseen things cause this to change, a lot of speculative money that was piggybacking on this cash flow might leave. This has happened in other times and places. Indeed, the price index in some of these super hot markets like SF and Seattle have seen sharp declines the last few months. You may hand wave this away by claiming that 'it always comes back' but that again doesn't actually calculate through how much opportunity cost you are paying by not pumping money into the market during down turns.
Another source of volatility is the inherent linkage between house prices and employment in the area. By buying a house close to where you are currently making a high salary (which you would want to since commute times disproportionately affect happiness) you are making a long term bet which, if it doesn't pan out, will simultaneously leave you without that high paying job, either forcing you into a long commute or selling the house and moving to a different city. This is extraordinarily common. Labor mobility boosts earnings, and it is difficult to account for this in a calculator, but the younger you are the more wary you should be of locking yourself out of major career moves that move you geographically.
'But the market is expensive too!' there are a couple important points about that. The first is that none of the typically cited measures that people tout to claim the market is expensive have historically been very good predictors. Remember, if you can make a confident prediction about the market you can make gigabucks, so anyone claiming this who isn't sitting on top of a mountain of money should be considered suspect. Second, let's look at three examples of people trying to time the market. All three people invest at a rate of $200/mo. Person A has the absolute worst luck in the world and invests on the eve of all the worst stock market crashes over the last 40 years. Person B has the best luck in the world and invests exactly at the bottom of all the crashes. Person C believes it is hard to out predict the market and just puts some of their savings into the market each month. Maybe you're guessing that person B and C do about the same even though C's strategy is way easier. Nope, it's even worse than that. Person A, the unlucky one wound up with 660k. Person B, the lucky one made 950k. But Person C, who just plugs $200 in rain or shine made...1.38 million. What? How is that possible? Person B still misses out on gains over time by waiting around with cash in hand trying to time things.
So let's plug this back into the housing situation. If a mere $200 a month over time in the market generated 1.38 million...well that puts a big damper on the stories of people making millions from holding a house for a long time. If you were to sit down and carefully go over the numbers with such happy folk they usually did worse than they would have putting similar amounts away into the market (people will get very mad at you if you do this). Again, people who happened to buy into the hottest markets before they sharply appreciated may have outperformed, but you shouldn't expect to be able to predict that ahead of time.
Aren't I doing the same thing from the other end by predicting that the stock market will do well over the next 40 years? It's very different because, as mentioned before, a single house is more like a single stock than it is like a globally diversified portfolio. When you look at a stock index over the long term, even things as catastrophic as the World Wars is just a blip. Betting on the world economy is just a much different bet than betting on a single house in a single market.
What about if I do something clever like buy a big house with my friends? Bigger houses cost less per room etc. A few problems. Going in on a house with multiple people also means splitting the tax advantages of home ownership. More luxurious homes are subject to bigger swings in home valuation during downturns than single family houses since that's where most of the demand is. Second, this situation multiplies the opportunity cost of not being mobile. The chance that at least some of the people will wind up needing to move sooner than expected is high. Which leads to the third thing which is that this is a much bigger hassle to coordinate both immediately and ongoing and it may end your friendships, which is going to be painful if you are living together for years.
What if I time the housing market? This doesn't work for the same reason that timing the stock market doesn't work. Pretend it is right after a crash and you're thinking of making an offer on a house. Where does the down payment come from? You can bet that lending standards have gone up in a crash and you're not going to get a zero down loan. The money will have to come from your investment portfolio, which means you're pulling money out of the market at the worst possible time to gamble on a particular house outpacing the market. This is a bad plan.
Come on! There has to be some situations in which buying makes sense! I did manage to find two situations where you can have a reasonable expectation of coming out ahead, or at least not doing too badly if things turn south. These scenarios are still highly reliant on you being sure you want to stay in a particular spot for at least 10 years, so there's opportunity cost and risk there. 3 Bedroom, 2 bath houses are in highest demand, leaving 2 and 4 bedroom places slightly better deals. If you buy either a 2 bedroom condo and rent the second bedroom to someone reliable, or rent a 3 bed/2 bath with a fourth bedroom converted garage and live in the converted unit while renting the house to a family, both of these can pay off while giving you future flexibility to not need to move if family planning comes along. In either case you just stop renting the extra room. In the case of the 4 bedroom you'd start renting the garage unit when you moved into the house until you needed it as well. If you want to pull off this strat in a crazy housing market you'll still need to find a good deal and run the numbers yourself. And you'll have a tendency to put your thumb on the scale to get the outcome you want, so you have to be pretty careful about that. Also, trying to convert a garage yourself can be a real time and money sink without guarantee of success due to insane zoning laws, so that kind of setup is rare. Using a strat similar to this you can stretch to maybe make rent to buy ratios of 25-30 close to break even i.e. in cities you'd actually want to live. The Bay Area is probably still out though unless you want to live far from your job which is a terrible misery inducing decision.
'I still think I should go in on a house with someone else'. One of the reasons the above scenarios work is that they don't introduce the complications of multiple stake holders. Having renters vs owners means the friction is relatively low if (more like when) things change.
Gambling on hot housing markets isn't a free lunch. It takes up a lot of your time (both before and after the purchase) and encourages poor emotionally driven decisions. And then it exposes you to a bunch of politics that will further degrade your quality of life as you now have a huge proportion of your net worth tied up in this scheme. The realtor market is oriented around juicing whatever misconceptions you have that will encourage you to buy whatever you can afford, and the politics around trying to force homes to be 'good investments' are toxic.
Okay so this is a lot of doom and gloom. Gee thanks, Romeo. Well, the reason I went to the trouble of writing all this is because there's a massive upside to being aware of these issues. And that's freedom from worrying that renting is bad! This pays huge dividends over time as you don't need to ever worry about locking anything down. Combine with having few enough belongings that moving is no longer a huge chore and the ability to move once every few years becomes a real boost not only to your career potential but also to not calcifying into routines that make your life seem shorter and less varied.