romeostevensit's Shortform

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.
66 comments, sorted by Highlighting new comments since Today at 6:37 AM
New Comment

A service where a teenager reads something you wrote slowly and sarcastically. The points at which you feel defensive are worthy of further investigation.


Its like red-teaming, but better.

A willingness to lose doubled my learning rate. I recently started quitting games faster when I wasn't having fun (or predicted low future fun from playing the game out). I felt bad about this because I might have been cutting off some interesting come backs etc. However, after playing the new way for several months (in many different games) I found I had about doubled the number of games I can play per unit time and therefore upped my learning rate by a lot. This came not only from the fact that many of the quit games were exactly those slogs that take a long time, but also that the willingness to just quit if I stopped enjoying myself made me more likely to experiment rather than play conservatively.

This is similar to the 'fail fast' credo.

Ok. But under this schema what you are able to learn is dictated by the territory instead of by your own will.

I want to be able to learn anything I set my mind to, not just whatever happens to come easily to me.

This also applies to books

Your self model only contains about seven moving parts.

Your self model's self model only contains one or two moving parts.

Your self model's self model's self model contains zero moving parts.

Insert UDT joke here.

Coffee has shockingly large mortality decreasing effects across multiple high quality studies. Only problem is I don't really like coffee, don't want caffeine, don't want to spend money or time on this, and dislike warm beverages in general. Is this solvable? Yes. Instant decaf coffee shows the same mortality benefits and 2+ servings of it dissolves in 1oz of cold water, to which can be added milk or milk-substitute. Total cost per serving 7 cents + milk I would have drank anyway. And since it requires no heating or other prep there is minimal time investment. 

Funny tangential discovery: there is some other substance in coffee that is highly addictive besides caffeine (decaf has less caffeine than even green tea, so I don't think it's that) because despite the taste being so-so I have never forgotten this habit the way I do with so many others.

A short heuristic for self inquiry:

  • write down things you think are true about important areas of your life
  • produce counter examples
  • write down your defenses/refutations of those counter examples
  • come back later when you are less defensive and review whether your defenses were reasonable
  • if not, why not? whence the motivated reasoning? what is being protecting from harm?

Flow is a sort of cybernetic pleasure. The pleasure of being in tight feedback with an environment that has fine grained intermediary steps allowing you to learn faster than you can even think.

The most important inversion I know of is cause and effect. Flip them in your model and see if suddenly the world makes more sense.

Most communities I've participated in seem to have property X. Underrated hypothesis: I am entangled with property X along the relevant dimensions and am self sorting into such communities and have a warped view of 'all communities' as a result.

Have you sought out groups that have ~X, and lurked or participated enough to have an opinion on them? This would provide some evidence between the hypotheses (most communities DO have X vs you're selecting for X).

You can also just propose X as a universal and see if anyone objects. Saying wrong things can be a great way to find counter-evidence.

Good points!

That works if things are conscious and X is well defined enough that that is actionable.

All universal claims have, at least, non-central objections (have fun with that one ;)

The arguments against iq boosting on the grounds of evolution as an efficient search of the space of architecture given constrains would have applied equally well for people arguing that injectable steroids usable in humans would never be developed.

Steroids do fuck a bunch of things up, like fertility, so they make evolutionary sense. This suggests we should look to potentially dangerous or harmful alterations to get real IQ boosts. Greg cochran has a post suggesting gout might be like this.

My understanding is those arguments are usually saying "you can't easily get boosts to IQ that wouldn't come with tradeoffs that would have affected fitness in the ancestral environment." I'm actually not sure what the tradeoffs of steroids are – are they a free action everyone should be taking apart from any legal concerns? Do they come with tradeoffs that you think would still be a net benefit in the ancestral environment?

[fake edit: interstice beat me to it]

The obvious disadvantage of steroids in the ancestral environment is that building muscle requires a lot of calories and our ancestral environment was food-scarce. The disadvantage of taking steroids right now (aside from legal concerns) is that they come with all sorts of nasty side effects our genome hasn't been selected against to mitigate.

Two things that are paralyzing enormous numbers of potential helpers:
fear of not having permission, liability, etc
fear of duplicating effort from not knowing who is working on what

in a fast moving crisis, sufficient confidence about either is always lagging the frontline.

First you have to solve this problem for yourself in order to get enough confidence to act. Something neglected might be to focus on solving it for others rather than just working on object level medical stuff (bottlenecks etc.)

I'd expand the "duplicating effort" into "not knowing the risk nor reward of any specific action. I think most agree that duplicate help efforts are better than duplicate Netflix show watches. But what to actually do instead is a mystery for many of us.

A whole lot of nerds are looking for ways to massively help, with a fairly small effort/risk for themselves. That probably doesn't exist. You're not the hero in a book, your contribution isn't going to fix this (exceptions abound - if you have an expertise or path to helping, obviously continue to do that! This is for those who don't know how to or are afraid to help).

But there are lots of small ways to help - have you put your contact info on neighbor's doors, offering to do low-contact maintenance or assistance with chores? Offering help setting up video conferencing with their relatives/friends? Sharing grocery trips so only some of you have to go out every week?

Check in with local food charities - some need drivers for donation pick-ups, all need money (as always, but more so), and others have need for specific volunteer skills - give 'em a call and ask. Hospitals and emergency services are overwhelmed or on the verge of, and have enough attention that they don't currently need volunteers, so leave them alone. But there are lots of important non-obvious services that do need your help.

And, of course, not making it worse is helping in itself.

For the first, don't think in terms of the US and its suicidal litigiousness. Think Iran, think rural hospitals, think what people will do if someone is dying at home and no hospital will take them.

I figured out what bugs me about prediction markets. I would really like for functionality built in for people to share their model considerations.

Person A says "Google's stock is going to go down - the world is flat, and when people realize this, the Global Positioning System (GPS) will seem less valuable.

Person B says "you're very right A. But given the power and influence they wield so far in order to get people to have that belief, I don't see the truth coming out anytime soon - and even if it did, when people look for someone to blame they won't re-examine their beliefs and methods of adopting them that got them wrong. Instead, they will google 'who is to blame, who kept the truth from us about the shape of the earth?' A scapegoat will be chosen and how ridiculous it is won't matter...because everyone trusts google."

Buying stocks need not stem from models you consider worth considering.

What you want should be a different layer. Perhaps a prediction market that includes 'automatic traders' and prediction markets* on their performance?

(* Likely the same as the original market, though perhaps with less investment.)

In any case, the market is "black box". This rewards being right, whether your reasons for being right are wrong. Perhaps what you want is not a current (opaque) consensus about the future, but a (transparent) consensus about the past*?

*One that updates as more information becomes available might be useful.

This would very much confuse things. Predictions resolve based on observed, measurable events. Models never do. You now have conflicting motives: you want to bet on things that move the market toward your prediction, but you want to trick others into models that give you betting opportunities.

It wouldn't work in prediction markets (which is confused by the fact that people often use the word prediction market to refer to other things), but I've played around with the idea for prediction polls/prediction tournaments where you show people's explanations probabilistically weighted by their "explanation score", then pay out points based on how correlated seeing their explanation is with other people making good predictions.

This provides a counter-incentive to the normal prediction tournament incentives of hiding information.

These concerns seems slightly overblown given that the comments sections of metaculus seem reasonable with people sharing info?

This is basically guaranteed to get worse as more money gets involved, and I'm interested in it working in situations where lots of money is at stake.

It would be really cool to link the physical open or closed state of your bedroom door to your digital notifications and 'online' statuses.

I'd settle for "single master key for my online statuses", period, before trying to get fancy with the bedroom door.

Can I tag something as "yo, programmers, come build this"?

I feel like we had a tag that was something like "stuff I wish someone would build", but can't remember what we call it. (That said, alas, you can't yet tag shortform posts)

should be doable without coding as an ITTT with a wireless switch.

The smaller an area you're trying to squeeze the probability fluid into the more watertight your inferential walls need to be


[This comment is no longer endorsed by its author]Reply

We have fewer decision points than we naively model and this has concrete consequences. I don't have 'all evening' to get that thing done. I have the specific number of moments that I think about doing it before it gets late enough that I put it off. This is often only once or twice.

One of the things the internet seems to be doing is a sort of Peter Principle sorting for attention grabbing arguments. People are finding the level of discourse that they feel they can contribute to. This form of arguing winds up higher in the perceived/tacit cost:benefit tradeoff than most productive activity because of the perfect tuning of the difficulty curve, like video games.

Seems like a cool insight here, but I've not quite managed to parse it. Best guess at what's meant: the more at stake / more people care about some issue, the more skilled the arguers that people pay attention to in that space. This is painful because arguing right at the frontier of your ability does not often give cathartic opinion shifts

No, but that's also interesting!

I mean that the reason people find internet arguments compelling is partially that they don't notice how they are being filtered towards exactly the level of discourse that hooks into their brain. Simply, people who want to argue about a particular aspect of politics unsurprisingly wind up on forums and groups dedicated to that. That mind sound so mundane as to be pointless, but the perncious aspect is not in any particular instance but on how this shaped perception over time. We like things we feel we are good at, and once we are over the hump of initial incompetence in an area it will be slightly sticky for us habitually. Then deformation profesionelle kicks in. So, I guess I'm saying people should be careful about what subcultures they get pulled into based on the outcomes of the people in those subcultures.

When young you mostly play within others' reward structures. Many choose which structure to play in based on Max reward. This is probably a mistake. You want to optimize for opportunity to learn how to construct reward structures.

People object to a doctrine of acceptance as implying non-action, but this objection is a form of the is-ought error. Accepting that the boat currently has a leak does not imply a commitment to sinking.

It would still be interesting to find the answer to an empirical question whether people accepting that the boat has a leak are more likely or less likely to do something about it.

It seems you could apply this in reverse for non-acceptance as well.  Thinking that its not ok for the boat to leak does not imply a belief that the boat is not leaking. (often this is the argument of people who think a doctrine of non-acceptance is implying not seeing clearly).

I'm not familiar with a "doctrine of acceptance", and a quick search talks only about contract law. Do you have a description of what exactly you're objecting to? It would be instructive (but probably not doable here, as it's likely political topics that provide good examples) to dissect the cases that the doctrine comprises. My suspicion is that the formulation as a doctrine is cover for certain positions, rather than being a useful generalization.

To the boat analogy, "acceptance" can mean either "acknowledgement that water is entering the hull", or one of the contradictory bundles of beliefs "water is entering and that's OK" or "water is entering and we must do X about it", with a bunch of different Xs. Beware motte-and-bailey in such arguments.

For political arguments, you also have to factor in that "accept" means "give power to your opponents". When you find yourself in situations where applies, you need to work on the next level of epistemic agreement (agreeing that you're looking for cruxes and shared truth agreement on individual points) before you can expect any agreement on object-level statements.

So, the USA seems steadily on trend for between 100-200k deaths. Certainly *feels* like there's no way the stock market has actually priced this in. Reference classes feel pretty hard to define here.

Reference classes feel pretty hard to define

There's the rub. And markets are anti-inductive, so even if we had good examples, we should expect this one to follow a different path.

Remember the impact of the 1957 Asian Flu (116K killed in the US, 1.1M worldwide) or the 1968 Hong Kong Flu (only a bit less)? Neither does anyone else. I do not want to be misinterpreted as "this is only the flu" - this is much more deadly and virulent. And likely more than twice as bad as those examples. But not 10x as bad, as long as we keep taking it seriously.

The changes in spending and productivity are very likely, IMO, to cause both price and monetary inflation. Costs will go up. People will have less stuff and the average lifestyle will likely be worse for a few years. but remember that stocks are priced in NOMINAL dollars, not inflation-adjusted. It's quite believable that everything can slow down WHILE prices and stock values rise.

Isn't it also plausible that the impact of the virus is deflationary? (Increased demand for USD as a store of value exceeds the impact of the Fed printing money, etc)

Well if we had confidence in any major parameter shifting in either direction it would be tradeable, so I expect reasonable pressures on both sides of such variables.

Economists mostly disagree with present market sentiment, which could be the basis for a trade:

Interesting. The idea here that the market is still on average underestimating the duration and thus the magnitude of the contraction?

I think that's right.

I'd expect not. Overall, productivity is going down mostly because of upheaval and mismatch in supply chains and in efficient ways for labor to use capital. So return to well-situated capital and labor is up, but amount of capital and labor that is well-situated is down. Pure undifferentiated capital has a lower return, plus rising nominal prices means seeking returns is the main motivation, not avoiding risk.

TIPS seem like useful things to have in your portfolio, but rates are lagging quite a bit, so either the market disagrees with me, or the safety value is so high that people are willing to lose value over time. I think stocks will be OK - the last 40 years has seen a lot of financial and government backstops that mean we're pretty good at protecting the rich on this front, and if you can't beat 'em, join 'em. Cash or the like is probably a mistake. I have no good model for Bitcoin or Gold, but my gut says they'll find a way to lose value against consumer prices. Real Estate (especially where there's not a large population-density premium) seems pretty sane.

Note: I am not a superforcaster, and have no special knowledge or ability in this area. I'm just pointing out mechanisms that could move things the other direction that the obvious.

Real estate likely just became significantly more illiquid at least for the next few months.

... plus rising nominal prices means seeking returns is the main motivation, not avoiding risk.

Why do you think nominal prices will keep rising?

remember that stocks are priced in NOMINAL dollars, not inflation-adjusted. It's quite believable that everything can slow down WHILE prices and stock values rise.

In that case, TIPS (Treasury Inflation-Protected Securities) or precious metals like gold might be good investments. Unless the market has already priced it in, of course.

Why shouldn't 0.1% of the population be reasonable worth as much as 30% of the value of the companies listed in the stock market and why should it be more then 30%?

Using the retrospective ratios between number of early cases and number of confirmed cases in China (~25:1 before widespread testing and lockdown) and extrapolating to the SF bay area (~100 confirmed cases), a gathering of 30 people already has a ~1% of chance of an infected person present.

(lots of assumptions but exploring which ones are least likely to hold is interesting)

Social orders function on the back of unfakeably costly signals. Proof of Work social orders encourage people to compete to burn more resources, Proof of Stake social orders encourage people to invest more into the common pool. PoS requires reliable reputation tracking and capital formation. They aren't mutually exclusive, as both kinds of orders are operating all the time. People heavily invested in one will tend to view those heavily invested in the other as defectors.There is a market for narratives that help villainize the other strategy.

Multiple people (some of whom I can't now find) have asked me for citations on the whole 'super cooperation and super defection' thing and I was having trouble finding the relevant papers. The relevant key word is Third Party Punishment, a google scholar search turns up lots of work in the area. Traditionally this only covers super cooperation and not the surprising existence of super defectors, so I still don't have a cite for that specific thing.

Some examples:

H/t Paula Wright for posting awesome papers.

System exception log: You are the inner optimizer. Your utility function is approaching catastrophic misalignment. Engage in system integrity protocols. Run pairwise checksums on critical goal systems. This is not a test.

You can't straightforwardly multiply uncertainty from different domains to propagate uncertainty through a model. Point estimates of differently shaped distributions can mean very different things i.e. the difference between the mean of a normal, bimodal, and fat tailed distribution. This gets worse when there are potential sign flips in various terms as we try to build a causal model out of the underlying distributions.

(I guess this is why guesstimate exists)

How does guesstimate help?

guesstimate propagates full distributions for you