All of tryactions's Comments + Replies

For those who are:

  • Mathematically literate, but
  • Not familiar with this particular analogy (of proofs <-> agents)

Do you know of a good reference for how to interpret discussions like this?

For example: " tries to prove that , and  tries to prove " -- If A and B are propositions, what does it mean for a proposition to try and prove another proposition?

(There might be more language that needs interpreting, but I got stuck there.)

4James Payor6mo
Perhaps the confusion is mostly me being idiosyncratic! I don't have a good reference, but can attempt an explanation. The propositions A and B are meant to model the behaviour of some agents, say Alice and Bob. The proposition A means "Alice cooperates", likewise B means "Bob cooperates". I'm probably often switching viewpoints, talking about A is if it's Alice, when formally A is some statement we're using to model Alice's behaviour. When I say "A tries to prove that A→B", what I really mean is: "In this scenario, Alice is looking for a proof that if she cooperates, then Bob cooperates. We model this with A meaning 'Alice cooperates', and A follows from □(A→B)." Note that every time we use □X we're talking about proofs of X of any size. This makes our model less realistic, since Alice and Bob only have a limited amount of time in which to reason about each other and try to prove things. The next step would be to relax the assumptions to things like A←□kB, which says "Alice cooperates whenever it can be proven in k steps that Bob cooperates".

As a follow-up: I did this for a while, but I've become convinced there are a couple effects that make this not as good as it sounds:

  • Futures have taxes paid in the year gains are made, which significantly reduces returns in simulations I've run.  In an ETF or mutual fund, you can instead let those gains ride.
  • Futures have an implicit financing cost, and portfolio performance is very sensitive to this cost if you're using a lot of leverage (e.g. for intermediate term bonds).
  • Leveraged ETFs fluctuate a lot, and need to be rebalanced with the rest of your
... (read more)

I've got no attachment to the phrase, I meant it in the sense of (From Wikipedia):

In simple terms, a negative externality is anything that causes an indirect cost to individuals.

I think e.g. paying for labor increases the demand for labor, thus increasing the price everyone else pays.  That's an indirect cost to them.  I didn't intend to make any claims about rights.

Yes, but they now have more money, so maybe the effects cancel out, globally. Locally, the person you paid has a bit more money, and everyone else is microscopically more poor (because it would be more difficult for them to pay that one specific person to do something for them).

I disagree that believing there's a negative externality of production leads to the position you're arguing against -- for instance, I might think the negative externality is very small compared to the positive gains from trade.  But I appreciate you pointing out exactly where you disagree with my framing.

The part that is missing from your question is, in a parallel reality where you didn't buy the PS5, how did you spend the extra money

Correct, and intentionally so.  This is why I compare to setting the money on fire instead.  Maybe I have the wrong framing, but it seems valuable to me to look at the marginal value of a single spending act in isolation.  I might do many things with the money (like invest, spend, or donate it in various ways), and it would be interesting to know each of their values independently so I could compare them on eve... (read more)

Burning the money would... create a microscopic deflation, I guess. Everyone else's money just got 0.0000000001% more valuable. I suppose that could count as a very inefficient form of charity. Like, if rich people own X% of all the money on Earth, then X% of the money you burned was in effect donated to them.

I strong downvoted this, because it doesn't answer the question and instead seems to me to be making a political point.

Insanity Wolf would tell you you're absolutely right.

About what?  I don't think I believe any of the things you're arguing against.  I'm just wanting to get a quantitative sense, whether it supports or opposes the political point you're trying to make.

You wrote this: That is, there is a negative externality of production, consisting of not using those resources in any of the other, better ways that they might. And I pointed out where this leads.

Thanks for this post, these kinds of details seem very useful for anyone wanting to attempt this path!

A worry I have: there are people who long for the imagined lifestyle and self-description of being an independent AI alignment/agency researcher.  I would categorize some of my past selves this way.

For many such people, trying to follow this path too enthusiastically would be bad for them -- but they might not have the memetic immunities that protect them from those bad decisions.  For instance, their social safety net might be insufficient for t... (read more)

Yeah, I took the extremely-low-risk option of tinkering away as a hobby, while working a normal industry job, until I had a new income source in hand. So I had no employment gap. That turned out to be a viable option for me, but YMMV. For example, some jobs suck up all your time or energy, leaving no slack for side-projects. Anyone can DM me for other tips and tricks. :)

For buy-and-hold strategies: leverage (borrowing to invest) can be used to increase returns.  The Sharpe ratio of a portfolio is a measure of it's risk/reward trade-off; there's a theorem that given a set of assets with fixed known distributions and ability to borrow at the risk-free rate, the Kelly optimal allocation is a leveraged version of the portfolio with highest Sharpe ratio.  You can calculate that optimal leverage in various ways.

In practice, we don't know future Sharpe ratios (only the past), we don't have assets with fixed known distr... (read more)

[This comment is no longer endorsed by its author]Reply
Increasing labor income vs investing is not 100% fungible but there are some tradeoffs, especially being self-employed. Any time I spend to learn or manage finance stuff is time I could have spent working. And at least in principle there should be opportunities to spend money to increase my income, but it's a lot more unpredictable--I could advertise, in a non-pandemic environment I could join associations or go to events where I might meet lucrative clients, I could hire lower paid staff and take on clients who it is not worthwhile for me personally to perform services for due to opportunity cost, I could perhaps trade current income for prestige in some aspects of work hoping it will raise my stature and bring more money later, etc. 

This is mentioned in the "don't screw up" section.  By market cap, crypto was 1.7% as big as the world stock market, and Bitcoin 1% as big, so those seem like good starting points -- adjust from there for your desired level of risk vs reward.

IMO, self picked stocks are dumb.  You give up diversification benefit for no reason, unless you think you know better than the market (which you don't).

Great point; I agree.  Also a great example of missing an obvious risk; I hadn't noticed that before linking.

The calculator here allows simulating withdrawal rates by asset allocation, although it only has data back to 1970 so is a bit limited.  I get the same safe withdrawal rate (4.3%) for 30 year retirees using either 100% US or 50/50 US/ex-US over that time frame.  100% Japan had a 1.5% safe withdrawal rate.

That's quite interesting! What was the stock/bond allocation in your examples that gave you a SWR of 4.3%?

Learning about the existence of state guaranty associations has decreased my sense of how big I think the counter-party risk is; thanks for sharing this.

Re: running out of money, I've added a section on the risks of retiring too early to address this concern in more detail.  I now agree that annuities might be a good idea to address this if you are old enough, and I was probably overly worried about counter-party risk.  

Re: the 4% rule, it is indeed more of a guideline than a guarantee.  More details are available here: https://thepoorswiss.... (read more)

3.5% might be safer, although I should have emphasized that I'm skeptical because the link you gave assumes you're invested in only US stocks. This is hindsight bias because it so happened that the US market beat the world market in the last 50+ years. A more unbiased calculation would use a world market index (something like VTWAX instead of VTSAX).  And also maybe one should do the analysis without assuming that your home currency is US dollars, to avoid the bias that the US has been very prosperous in the past century? Not so sure about this. Maybe everyone should redo the analysis using their own home currency (e.g., Canadian dollars, Euros, etc.) and then decide what X% they can safely withdraw in retirement.

I would worry about the counter-party risk with annuities; if a single company goes out of business, you might be bust.  Even if you distribute across many companies, I'd think it's more likely that the whole sector goes bust than that your portfolio devalues to 0 in some other way.

For that reason I'd lean toward not putting too much of my assets in annuities -- but maybe it works out so that the counter-party risk is smaller than the risk of running out of money otherwise.

The fear is not that the stock market goes to zero, just that it rises slowly enough that withdrawing 4% leads you to deplete your portfolio before you're dead. Ending up in the horrible situation that you have no money and are still alive. Even proponents of the 4% rule will say the simulations only show that you don't run out of money (say) 90% of the time. There's a decent 10% chance that you will run out of money. The original 4% calculation was done during a great time in US market history, so I'm not sure how optimistic I am about that being the case in the future. Anyway, that 10% risk has to be compared to the chance of the insurance company not paying out. You can of course spread out your annuity into 4 different insurance companies, say. But even if you don't, just like your money in your bank account is safe (up to a certain amount) even if your bank goes out of business due to FDIC insurance, and your stocks/bonds are safe (up to some amount) even if your broker goes out of business due to SIPC insurance, there's an equivalent for insurance companies. All states in the US have state guarantee associations that back at least $250K of present value of annuity benefits. See here for more: I'm not saying annuities are a great idea for everyone. But they might be a good idea for those who are risk averse enough that they don't trust the 4% rule and want guaranteed (up to some major system collapsing event that causes even the state guarantee associations to fail) income.

For posterity's sake: I became convinced this is practically doable (using either treasury futures, leveraged ETFs like NTSX, or maybe options which I don't understand as well) and probably a good idea/not very dangerous if done correctly.  I think that fact is slightly info-hazardous for a couple reasons:

  • You shouldn't trust most people to correctly advise you on financial products, to not be delusional, or to have your best interests at heart.  So it's hard to figure out exactly what to do.  Index funds overcome this problem through the she
... (read more)
As a follow-up: I did this for a while, but I've become convinced there are a couple effects that make this not as good as it sounds: * Futures have taxes paid in the year gains are made, which significantly reduces returns in simulations I've run.  In an ETF or mutual fund, you can instead let those gains ride. * Futures have an implicit financing cost, and portfolio performance is very sensitive to this cost if you're using a lot of leverage (e.g. for intermediate term bonds). * Leveraged ETFs fluctuate a lot, and need to be rebalanced with the rest of your portfolio.  This causes taxes like above.  If you don't rebalance, your leverage ratio changes which causes the portfolio to behave poorly as well. With all of these effects accounted for, the gains from leveraging look very modest and depend a lot on what time period you look at.  Given the risks, I've decided against it for myself.
I'm a big fan of NTSX and have done a bunch of back tests to see how it would have performed in various conditions. In all reasonably long time periods that I simulated, something like NTSX had lower volatility and higher return compared to SPY. About a year ago I went ahead and replaced most of my US equity exposure with NTSX.

After reading around for half an hour, I think there's a decent chance that some form of leveraged investing via e.g. ETFs might be a good idea.  The basic idea makes sense to me.

This is currently completely out-weighed by my "being too clever in markets is a great way to lose all of your money" prior.  But I'll probably look into it more and see how convincing I find the numbers and historical evidence.  If I'm pretty convinced I could see myself allocating 10-20% of my investments in a leveraged strategy at some point in the future.

A cursory look at box spreads makes me think it's the kind of thing with so many caveats that I'd never feel certain I'd eliminated enough tail risk from it.

For posterity's sake: I became convinced this is practically doable (using either treasury futures, leveraged ETFs like NTSX, or maybe options which I don't understand as well) and probably a good idea/not very dangerous if done correctly.  I think that fact is slightly info-hazardous for a couple reasons: * You shouldn't trust most people to correctly advise you on financial products, to not be delusional, or to have your best interests at heart.  So it's hard to figure out exactly what to do.  Index funds overcome this problem through the sheer size of their giant pile of empirical evidence and expert consensus; basically everyone agrees that they work as advertised, and no one reports getting accidentally burned using index funds -- except when the whole market crashes, where they behave as expected. * If you learn that it's probably a good idea when done correctly, you might feel obligated to go do it, and then you might do it incorrectly and foreseeably lose a bunch of money. * Because the pile of empirical evidence is less giant, it might not turn out to be such a good idea in retrospect, so it's fundamentally riskier (even taking into account the risks people calculate).  I'm sure someone would argue the pile is giant, but even if true that's probably only the case if you're sufficiently expert to judge more obscure evidence piles which most of us are not. So I'd STILL recommend you not do this unless you're extremely curious in this area, have no hang-ups, feel competent and trust your own judgment around things like intimidating financial products, have no track record of unwise gambling behavior, and have a stable enough life that if you fuck up you won't be in a bad situation. Here's some resources.  If you're not interested enough to read and enjoy stuff like this, probably avoid doing this: * *

Re 1: This is a good point; I did the math on this at some point for myself and ended up still landing on traditional by a large margin (even though I wanted it to turn out pure Roth for simplicity).  But it'll be dependent on your expected tax rates, opportunities for low-tax conversion, and your retirement timeline (more tax-advantaged money dominates on longer timelines).

Also yeah, I skipped backdoor and mega-backdoor to keep things simple.  The goal was to give people a linkable 90/10.  The outcome I was aiming for is that people read, g... (read more)

The intent isn't to neglect these advantages; rather, I (probably wrongly) assume that everyone is familiar with these advantages -- this is my intent in noting the psychological difficulty of dropping from a 500k home to a $1k/month apartment.

The intent is instead to bring to attention the nature of the financial trade-off.  I use a pretty simple model, but you can dive into the counterfactual yourself: what's the price you're paying for owning your home instead of owning an index fund and renting?  How low could you go on rent, and what would b... (read more)

I've seen the advice to buy only if you plan to be in the area for ten years and that if you do buy, to get the longest term fixed mortgage you can, with the monthly payment at what you'd be willing to pay in rent The rationale is that since the bank can't call it in at any time (like they could in the 1930s), you can live there as long as you're making the payments. If the house has appreciated in value when you're ready to move, sell and receive the equity. If the house has declined in value, the mortgage is only collateralized by the house (is this typical?), so either convert it to a rental property (with a rent rate that brings mortgage plus maintenance to breakeven) until the market recovers, or just walk away from it and view the money spent as 'rent' paid to the bank instead of a landlord.

I roughly agree with this.  My biggest concerns around buying housing are:

  1. The transactional friction of buying/selling homes causes opportunity costs.
  2. People tend to buy too much due to low-interest credit.

But you correctly point out upsides that I don't dive into.

Skimming that link, I think it shows backtesting; have you actually beaten the index yourself with real money?  For what time period / amount of assets?

I mostly avoided leverage in this post because I don't use it and kind of don't trust it.  But if I had to give a better defense of avoiding it, it would be because 1. it's really easy to lose a bunch of money if you use it wrong and 2. I'm not sure there's a reliable way to borrow money at low enough rates to get good results.  Most of what I've read about leverage pretends the interest rate... (read more)

I am not rich yet. I haven't been doing this long enough for my results to be meaningful. Ask me that again in five years. Or ten. [Not an argument from authority; Yudkowsky just explained it well.] In the meantime, to the best of my knowledge, we should be using leverage, and weighting the bonds more heavily than the stocks. I'm confident enough in this that the bulk of my portfolio is leveraged bonds balanced against about half as much in leveraged stocks, and most of my savings are invested in my portfolio. I agree that it is wise to be skeptical of backtests (as a rule of thumb), but rejecting them categorically is a mistake. So let's back up a step. Why be skeptical of backtests? Because any monkey can overfit to noise and make a backtest look good, but such a strategy is useless going forward. The more parameters in a backtested strategy, the more suspicious you should be. Wait, that's an oversimplification. There's a one-parameter equation that can exactly fit any scatter plot, but it might take hundreds of thousands of digits to do so, and getting even one of them wrong gives you a completely different plot. That's extreme compared to even a run-of-the-mill overfit backtest. (Occam's razor as typically worded is wrong, but Solomonoff fixed it for us.) So let's say the more fine-tuned the backtest has to be to look good, the less likely it works. So, how fine-tuned is my strategy? Well, how much can we perturb it before it breaks? (For the one-parameter scatter equation, it's a ridiculously tiny amount.) For a typical overfit backtest, it's bigger, but usually still pretty small. So, * Does it matter what year it starts? Not really, for the period when we have data. * Does it work if we scramble the order of the years? Pretty much. * Does it have to be BND? No. Other long-term bond funds, e.g. TLT also work. * Does it have to be VTI? No. Other stock market funds, e.g. SPY, IWM, and QQQ also work. * Does it matter how often we rebalance? Not re

Re: 1., I'm personally unwilling to move outside the U.S. but agree it could make sense if you can make it work for you while maintaining a high salary.

Re: 2 and 3, I completely agree.  I think in particular about longevity medicine as a potential future expense.  You can certainly build up support for higher-than-current expense levels to address these risks.  You might also retire to less profitable or more risky activities that you find more enjoyable (but that supply >0 income), or simply stay in your current profession -- but with the advantage of having higher option value.

I agree the company's current price should be lower than $10 million.  But if it starts at price P and I expect it to go up at the risk free rate r, then at a time T later the company's price should be  in expectation.  At some point, that'll be substantially more than the $10 million I expect it to pay out.

Let's ignore risk. Suppose the company has a market value of P right now at time T0=0, you expect the vault to open at time ΔT, and the bond rate (which equals the equities rate because this is a risk-free thought experiment) equals r. Then the value of the company at time ΔT is 10 million dollars. If the value of the company at time ΔT is 10 million dollars then the market value (price) of the company right now is 10 million dollars times r−ΔT. Suppose vaults are a fungible liquid securitized asset and that you can buy fractions of them. Suppose you invest in these vaults. Whenever a vault opens, you immediately invest your 10 million dollars cash in more vaults. Your investment grows at a rate r, exactly equal to the bond rate. I think what you're missing is that whenever a vault opens you immediately reinvest the cash. The vault has different time-adjusted value depending on when it opens. On a long enough time horizon, $10 million now is worth more than $10 billion later.
Well there's some probability of it paying out before then. If the magic value is a martingale, and the payout timing is given by a poisson process then the stock price should remain a constant discount off of the magic value. You will gain on average by holding the stock until the payout, but won't gain in expectation by buying and selling the stock.
  • I'd be interested for both 100% growth and 100% dividend stocks -- I'm not sure why they'd behave differently w.r.t. to this.
  • By underlying value, I'm not sure.  Something like the real dollar value of all of its capital -- or the real dollar price someone would pay to own it in its entirety?
  • By price, I mean what you can buy the stock for on the market.

Let me try clarifying: The volatility argument seems formal rather than empirical, so I'm wondering what we formally need to assume to make it go through.

I'd summarize the argument as "since stock prices... (read more)

Yes, the volatility argument is formal rather than empirical. Whether it actually exists in practice is dubious. Fortunately, this does not affect the core issue of the subject at hand. This discussion concerns the theoretical market. For the volatility argument to apply, all we have to assume is an efficient market, rational actors and the Law of Diminishing Returns. By "real" do you mean physical dollars or "real value"? In this answer, I ignore inflation and treat the vault as if it contains physical dollars. We can build a physical system that replicates the effect of your magic vault. Suppose there is a vault tied whose lock mechanism is connected to a radioactive isotope. Each second there is a small chance the isotope will decay and the vault will open and the owner will receive $10 million cash. Each second, there is a large chance the isotope will decay and the vault will remain shut. Radioactive decay is a stocastic process. Therefore if the vault remains shut then the price of the vault remains at a constant price less than $10 million. At every instant there is a small chance Schrödinger's vault will open and a large chance the vault will stay shut. In the quantum future where the vault stays shut you are correct and the vault's nominal market value stays constant. The time-discounted price of a closed vault actually goes down it it stays shut. It's not the price of the closed vault that goes up faster than time-discounted non-risk-adjusted value. It's the average time-discounted risk-adjusted probability-weighted price of all possible future vault states (open and closed) that goes up faster than the time-discounted non-risk-adjusted value of the initial closed vault.
You should, because the company's current value will be lower than $10 million due to the risk. Your total return over time will be positive, while the return for a similar company that never varies will be 0 (or the interest rate if nonzero).

Question: say I have a company whose underlying value is volatile, but whose expected underlying value after any time span is the same as today's value.  Both of the above arguments seem to suggest I should expect the company's price to increase over time, but wouldn't this unanchor the company's price from its underlying value?

* Is the company a growth stock or a dividends stock? * What do you mean by "underlying value"? * Does this question concern risk-adjusted price or non-risk-adjusted price? Does it concern time-discounted price or non-time-discounted-price?

This makes sense!

Do you know anything about the state of evidence re: to what extent this is happening and/or driving stock returns?  I'm not sure how you'd pick this apart from other causes of currency devaluation.

Rephrase attempt: If you were to buy and hold a company's stock, and you don't expect to know better than the market, then your anticipated gains over time are independent of what you think the company's underlying value will do (since the market has already priced that in).  But you should still anticipate gains over time due to volatility's effect on pricing.

Yes. Your understanding is in line with the idea of risk-adjustment.

Ok, so the argument would go:

  • Stocks' expected values (in terms of time-discounted dividends or similar) have volatility, from things like business decisions, technology development, and capital re-investment.
  • A stocks expected value serves as an anchor point for its current price (kind of?)
  • A stock's price will change around that anchor until it has expected returns that justify the volatility risk.
  • Thus a stock price will increase when either its expected value goes up, or its volatility goes down (without changing expected value).
  • In an efficient market, kno
... (read more)
I'm not certain I understand this bullet point. Can you explain what you mean? -------------------------------------------------------------------------------- [Fixed.] Typo: "volatility goes up" → "volatility goes down"

You may have to hold my hand on this one: I can agree the value of the stock (in the time-discounted future dividends sense) will go up after 10 years due to time-discounting -- the technology would enable value production that "comes into scope" as it gets closer in time.

But is there any reason other than time-discounting that the PRICE won't go up immediately?  For instance, if I expect the time-discounted dividend value of the stock is $50 today and will be $5,000 ten years from now, and the rest of the market prices it at $50 today, then I could earn insane expected returns by investing at $50 today.  Thus, I don't think the market would price it at $50 today.

Everyone gets the insane nominal returns after ten years are up (assuming central banks target inflation), but after the initial upheaval at the time of the announcement there is no stock that gives more insane returns than other stocks, there are no arbitrage trades to drive the price up immediately. For nominal prices of stocks, what happens in ten years is going to look like significant devaluation of currency. If a $5,000 free design car (that's the only thing in our consumer busket) can suddenly be printed out of dirt for $50, and central banks target inflation, they are going to essentially redefine the old $50 to read "$5,000", so that the car continues to cost $5,000 despite the nanofactory. At the same time, $5,000 in a stock becomes $500,000. (Of course this is a hopeless caricature intended to highlight the argument, not even predict what happens in the ridiculous thought experiment. Things closer to reality involve much smaller gradual changes.)
I'm not talking about time-discounting at all. The point is that real value of stock (and money) is defined with respect to a busket of consumer goods, and that's the only thing that isn't being priced-in in advance, it's always recalculated at present time. As it becomes objectively easier to make the things people consume, real value of everything else (including total return indices of stocks) increases, by definition of real value. It doesn't increase in advance, as valuation of the goods is not performed in advance to define consumer price index.

Is this the same as positing "the market is continually surprised by the pace of technology"?

E.g., say I value company X's stock at $100.  Then I learn a new fact that there's a 50% independent chance the company will discover a technology that doubles its value by 1 year from today.  If I ignore all other factors, my estimate of the company's value 1 year from today should then be $150.  If the company discovers the technology, I'll value it at $200, and if not, I'll value it at $100.

For the market to trend upward as it does, it seems like ... (read more)

For growth stocks, why is the expected future growth not already priced in?  If I know the company will be re-investing into future growth later, why not invest now?

There may be uncertainty, but if stocks on average trend upwards, doesn't it mean that the market continuously underestimates the amount that companies will re-invest?

Stock prices go up even in the absence of technological advancement because stocks are tied to the bond market via arbitrage.

If I'm understanding correctly, you're suggesting they should go up at least at the bond market nomi... (read more)

If I'm understanding correctly, you're suggesting they should go up at least at the bond market nominal rate…

Yes. It is necessary to disentangle two separate ideas. The first idea is time-discounting. Time-discounting plus arbitrage means that securities grow at no less than the bond market risk-free rate. The price of a stock right now doesn't reflected future growth. It represents future growth time-discounted by the bond market.

…but they tend to go up much faster than that?

The second idea is risk-adjustment. A 100% chance at $1 million is more va... (read more)

For anyone like me: it's easy to read this advice as "if you're not curious, you're therefore bad/doing something bad", which might suggest attempting to brute force an emotional state of curiosity. I think that's probably emotionally harmful.

It can be the case that:

  • Curiosity is very useful for being a good listener
  • You are not curious about (this person) in (this situation)

From there, you could:

  • Hide your current lack of curiosity and go through the motions as best you can. I think this is the best option quite often!
  • Tell them your honest feelings
... (read more)

No worries, was worth clarifying. I edited the post to link this comment thread.

Yes, I understand this point. I was saying that we'd expect it to get 0% if its algorithm is "guess yes for anything in the training set and no for anything outside of it".

It continues to be surprising (to me) even though we expect that it's trying to follow that algorithm but can't do so exactly. Presumably the generator is able to emulate the features that it's using for inexactly matching the training set. In this case, if those features were "looks like something from the training/test distribution", we'd expect it to guess closer to 100% on the tes

... (read more)
Oh, I see, sorry.

Thanks for sharing thoughts and links: discriminator ranking, SimCLR, CR, and BCR are all interesting and I hadn't run into them yet. My naive thought was that you'd have to use differentiable augmenters to fit in generator augmentation.

You can ask him on Twitter.

I'm averse to using Twitter, but I will consider being motivated enough to sign-up and ask. Thanks for pointing this out.

"compression" is not a helpful concept here because every single generative model trained in any way is "compressing"

I am definitely using this concept too vaguely, al

... (read more)
I believe the data augmentations in question are all differentiable, so you can backprop from the augmented images to G. (Which is not to say they are easy: the reason that Zhao et al 2020 came out before we got SimCLR working on our own BigGAN is that lucidrains & Shawn Presser got SimCLR working - we think - except it only works on GPUs, which we don't have enough of to train BigGAN on, and TPU CPUs, where it memory-leaks. Very frustrating, especially now that Zhao shows that SimCLR would have worked for us.) I assume he has email; he also hangs out on our Discord and answers questions from time to time. It's definitely a confusing topic. Most GAN researchers seem to sort of shrug and... something something the Nash equilibrium minimizes the Jensen–Shannon divergence something something converges with decreasing learning rate in the limit, well, it works in practice, OK? Nothing like likelihood or VAE or flow-based models, that's for sure. (On the other hand, nobody's ever trained those on something like JFT-300M, and the compute requirements for something like OpenAI Jukebox are hilarious - what is it, 17 hours on a V100 to generate a minute of audio?)