All of ryan_b's Comments + Replies

I am beginning to think that histories of mathematical struggle and failure are my favorite kind. One that is similarly a tale of challenging and repeated failures on an unintuitive subject is thermodynamics, and an amazing book on this subject is The Tragicomical History of Thermodynamics, 1822–1854 by Clifford Truesdell, himself a mathematical physicist most famous for continuum mechanics.

Alternative frame: I've been poking at the idea of quantum resource theories periodically, literally on the strength of a certain word-similarity between quantum stuff and alignment stuff.

The root inspiration for this comes from Scott Aaronson's Quantum Computing Since Democritus, specifically two things: one, the "certain generalization of probability" lens pretty directly liberates me to throw QM ideas at just about anything, the same way I might with regular probability; two, the introduction of negative probability and through that "cancelling out" pos... (read more)

Yes. The dominant ones are:

• Military experience: I was in the infantry for 5 years, and deployed twice. I gained an extremely high appreciation for several important things: the amount of human effort that goes into moving huge amounts of people and stuff from A to B; growth and decay in the coordination and commitment of a team; the mindblowingly-enormous gap between how a strategy looks from on high to how it looks on the ground (which mostly means looking at failure a lot).
• Jayne's macroscopic prediction paper: I am extremely libertine in how I apply thes

I watched that talk on youtube. My first impression was strongly that he was using hyperbole for driving the point to the audience; the talk was littered with the pithiest versions his positions. Compare with the series of talks he gave after Zero to One was released for the more general way he expresses similar ideas, and you can also compare with some of the talks that he gives to political groups. On a spectrum between a Zero to One talk and a Republican Convention talk, this was closer to the latter.

That being said, I wouldn't be surprised if he was sk... (read more)

Would it be correct to consider things like proof by contradiction and proof by refutation as falling on the generation side, as they both rely on successfully generating a counterexample?

Completely separately, I want to make an analogy to notation in the form of the pi vs tau debate. Short background for those who don't want to wade through the link (though I recommend it, it is good fun): pi, the circle constant, is defined as the ratio between the diameter of a circle and its circumference; tau is defined as the ratio between the radius of a circle and ... (read more)

Separately from gwern's argument, I say that maintaining the gap is still of vital national interest. As an example, one of the arguments in favor of nuclear testing bans is that it unilaterally favors American nuclear supremacy, because only the US has the computational resources to conduct simulations good enough to be used in engineering new weapons.

That logic was applied to Russia, but the same logic applies to China: advanced simulations are useful for almost every dimension of military competition. If they let advanced compute go, that means that the... (read more)

On the strategy engine paired with NLP: I wonder how far we could get if the strategic engine was actually just a constructed series of murphyjitsu prompts for the NLP to complete, and then it tries to make decisions as dissimilar to the completed prompts as possible.

My guess is that murphyjitsu about other players would be simpler than situations on the game map in terms of beating humans, but that "solving Diplomacy" would probably begin with situations on the game map because that is clearly quantifiable and quantification of the player version would route through situations anyway.

portraying Sam Bankman-Fried as the Luke Skywalker to CZ’s Darth Vader? Presumably that will change a bit.

I feel like Han Solo and Jabba the Hutt is the new Star Wars narrative, because it looks like SBF is going to owe some people.

Although I would also entertain just moving SBF to Lando for a sympathetic portrayal, in the name of making the Robot Chicken version real.

I agree the gradient-of-physical-systems isn't the most natural way to think about it; I note that it didn't occur to me until this very conversation despite acausal trade being old hat here.

What I am thinking now is that a more natural way to think about it is overlapping abstraction space. My claim is that in order to acausally coordinate, at least one of the conditions is that all parties need to have access to the same chunk of abstraction space, somewhere in their timeline. This seems to cover the similar physical systems intuition we were talking abo... (read more)

I agree two ants in an anthill are not doing acausal coordination; they are following the pheromone trails laid down by each other. This is the ant version of explicit coordination.

But I think the crux between us is this:

It seems to stretch the original meaning

I agree, it does seem to stretch the original meaning. I think this is because the original meaning was surprising and weird; it seemed to be counterintuitive and I had to put quite a few cycles in to work through the examples of AIs negotiating without coexisting.

But consider for a moment we had beg... (read more)

I am happy with longer explanations, if you have the time. To be more specific about the kind of things I'm interested in:

• Do you think Ukrainian forces are able to launch a campaign into Crimea?
• Do you think Russian forces are able to respond in time?
• If Russian forces do respond in time, do you think they will provide effective resistance?

In my model these kinds of questions tend to have a much bigger impact on diplomatic decisions than rhetorical or propaganda ones, and the recent history of the war has generated a lot more uncertainty about them through Russia's surprising underperformance.

The voting example is one of those interesting cases where I disagree with the reasoning but come to a similar conclusion anyway.

I claim the population of people who justify voting on any formal reasoning basis is at best a rounding error in the general population, and probably is indistinguishable from zero. Instead, the population in general believes one of three things:

1. There is an election, so I vote because I'm a voter.
2. Voting is meaningless anyway, so I don't.
3. Election? What? Who cares?

But it looks to me this is still coordination without sharing any ex... (read more)

What are your thoughts about the object level of the conflict in Ukraine and Russia, and what bearing do you think they have on the Crimea question?

Thinking about ways in which this safety margin could break; is it possible to have a thin mapping layer on top of your Physics simulator that somehow subverts or obfuscates it

I suppose that a mapping task might fall under the heading of a mesa-optimizer, where what it is doing is optimizing for fidelity between between the outputs of the language layer and the inputs of the physics layer. This would be in addition to the mesa-optimization going on just in the physics simulator. Working title:

CAIS: The Case For Maximum Mesa Optimizers

I don't think so, no - the way I understand it, any kind of separation into separate systems falls into the CAIS sphere of thought. This is because we are measuring things from the capability side, rather than the generation-of-capability side: for example, I think it is still CAIS even if we take the exact same ML architecture and train copies of it into different specializations which then use each other as services.

There are a couple of things worth distinguishing, though:

1. That a sufficiently integrated CAIS is indistinguishable from a single general age

That doesn't appear to be explained specifically, but what I think they are giving is the larger model size equivalence. That is to say, the 350M parameter language model with Mind's Eye is about as good as a 2.5B parameter language model, and so on.

Yeah, I shoulda linked that. Fixing shortly, thanks to niplav in the meantime!

Because the way they went about it was to give the language model access to a separate physics simulator (MuJoCo, owned by DeepMind) rather than something like the language model learning the rules of physics through a physics textbook or landing on some encoding of English tokens that happens to represent physics.

I interpreted having to go to a different engine to get better inputs for the language model as counting for multiple interacting services.

This sentence refers to interest groups, not to politicians or officials. It refers to unions, who want bargaining power, and union members, who want stable jobs and good pay. Or to businesses with a captive market, like American shipbuilders and dredge operators.

These groups think they are getting a payoff from status quo, and it is one they want to keep. The solution is therefore to match the payoff under the new proposal, or persuade them (of the truth that) they are not actually getting the payoff they think.

All of these examples seem like different variations of how to account for problem information.

I am reminded of a blog post about algorithms in scientific computing. Boo-hiss, I know, but - the claim of the blog post is that algorithmic efficiency is about problem information, and the more information the algorithm can capture about the problem the more efficient it can be. The example in support of the claim is the solving of linear systems of equations, and I establish relevance in this way: linear systems of equations are used in linear programming, whi... (read more)

In light of central explanations one and two, I feel like this fairly cries out for a reasoned rule in the Kahneman sense.

Short recap: this is the idea that you consult the best available experts to determine what the 6-8 most important dimensions of the problem are, and then give each of these a score, like cells in a spreadsheet. The default approach is to weight all these equally. The story goes this routinely meets expert performance even in fields where there are good experts, and tends to exceed it in fields where there isn't anything like real exper... (read more)

I can't resolve the disconnect about NATO completely, but: I find it works a lot better if we do not focus on NATO per se, but rather consider NATO to be the heading under which Putin's government talks about the basic geopolitical problems between Russia and Western Europe. For example, the Dugin school of thought views the European side of the security problem as being a fundamental one dictated by geography. For people in Russian leadership who subscribe to this notion, I expect the significance of NATO to them is as the current incarnation of a permanent problem.

My reaction is "Ha!" and/or "Ew."

This should also be transformed into a horror movie in the direction of Ring and Insidious, where each time we view the photos the malevolent spirit gets more integrated into our lives and when it shows up in a best friends forever photo alone with us it gets to assume our lives.

And enjoy a refreshing Coca-Cola, of course.

In the top-right corner of Lesswrong, you should be able to see your username. In my view it is third from the right, alongside a star for the karma summary and a bell for alerts.

If you click on your username, a drop-down menu should appear. On mine, the very first item is "New Question" which will open up a draft Question.

My wife's iPhone put together one of its Over the Years slideshows for my birthday, and she sent it to me because - exactly as the designers hoped - it contains several good memories. I have two wonderings about this.

First, the goodness in good memories. We periodically talk about the lowest-cost, best-return psychological intervention of gratitude, which seems heavily wrapped up with the idea of deliberately reviewing good memories. I know nothing about the algorithm that is used to put together these slideshows, but it seems to me it mostly triggers on v... (read more)

Plan A for my daughter is martial arts; the wife and I quibble a bit about what to look for when the time comes. I am much more tolerant of injury risks, on the grounds that no-contact martial arts doesn't advance the purpose of self defense. I would consider MMA; wife is opposed to MMA chiefly on cultural grounds.

I would also like her to do one of the team competition sports, by which I mean thing like soccer, basketball, and volleyball over track, wrestling, or swimming. The latter set is no different from totally individual competition, because there is... (read more)

You might have heard it described as "PeeWee", which means small children. In general, it refers to elementary school aged leagues for otherwise contact or equipment-intensive, like football and hockey. Elementary schools in the United States do not spend money on fields and equipment for these things, not least because they have playgrounds to maintain instead.

The blue cheese of comments is the funniest phrase I have read in weeks; I am looting it. If constructively contrarian arguments stink like a good cheese, then we might say the path to rational discourse is trod with the feet of god.

This year Petrov day almost sneaked past me. This strikes me as weird on account of the biggest proxy war since the 80s being underway, putting us closer the same stakes in realspace.

This comment is the first successful deployment of agree-disagree trick I have seen. Neat!

Ah, thank you - I didn't twig on the incentives comment, but I can see how that would be a signal of different operation.

I noticed you mention you work in one of these areas: from your perspective, what would you want an org like this to do differently from the existing ones that would make it more successful at getting policy implemented?

This project has real Carrick Flynn vibes: well-meaning outsider without much domain expertise tries to fix things by throwing crypto money (I assume) at political problems where money has strongly diminishing returns.

Can you talk a bit more about what gave you this vibe? They aren't starting a fund or a PAC, which is what comes to mind for me when people throw money at problems, and is literally what Carrick Flynn did.

I have a book on my to-read list on exactly this topic, for those interested: Artists of the Possible: Governing Networks and American Policy Change since 1945

I originally picked up the recommendation from a post at the Scholar's Stage.

Interesting - I interpreted this section differently, and yet I think it ultimately cashes out as agreeing with your comment about incentives.

In my reading, the clear concrete instructions are about the priorities, and about how to communicate. From the rest of the post I understood clearly that this means instructions like:

• Priority 1 this week is X. In any decision with a tradeoff between X and Y, choose X.
• Work on X for the next 4 hours after this meeting. Do not work on anything else.
• Schedule miscellaneous meetings on Tuesdays. Do not schedule them on an

Reviewing the examples in the post again, I think I was confused on first reading. I initially read the nuclear reactor example as being a completed version of the Michaelangelo example, but now I see it clearly includes the harms issue I was thinking about.

I also think that the Library of Babel example contains my search thoughts, just not separated out in the same way as in the Poorly Calibrated Heuristics section.

I'm going to chalk this one up to an oops!

Upvoted, because I think this a naturally interesting topic and is always relevant on LessWrong. I particularly like the threshold optimization section - it is accessible for people who aren't especially advanced in math, and sacrifices little in terms of flow and readability for the rigor gains.

I don't agree that the cost of a false-positive is negligible in general. In order for that to be true, search would have to be reliable and efficient, which among other things means we would need to know what brilliant looked like, in searchable terms, before we f... (read more)

This isn't spending per se; rather it is increasing costs. Any increase in spending happens in the course of existing programs, such as handing out more loans once students respond to the incentives.

On top of this, the federal financial structure is a unique and horrifying house of cards. Their accounting methods are actually unique to them, and sometimes vary by department or agency; auditing is difficult and inconsistent; much of it is seemingly designed to obfuscate, though in a change-the-standards-by-committee-to-make-us-look-less-bad way rather than ... (read more)

I wonder if someone were to form a credible educational institution that used income sharing agreements in lieu of various loans, whether it could directly out-compete the current university system.

I am not doing anything different from you, but I don't see any major tactical shifts that make much sense. The problem is that 401k and index funds already are the maximum-uncertain-future choices, for any future where the stock market succeeds as an institution. Residential real estate already is the lowest risk bet for any future where land is assessed according to price rather than according to use.

So mostly what I am trying to do is:

• Identify ways to make my property more useful. This is basic things, like growing a chunk of our food, increasing the am

I agree they make for really good stories. I tell you what I would like to see more of in these stories is leaning into the moral dessert of it all. 

• Fox and Hound: make friends and gain the ability to survive bear attacks!
• Mononoke: make not-enemies and not-die to spirit stampedes or cold iron!
• Primal: make friends and you can eat anything!

Actually, the Primal example is so on the nose I feel like a better term is needed for coordination-related-morality. Moral dinner seems fitting. Be good, so you can eat.

If we’re talking Mad Max: Fury Road, or even Beyond Thunderdome, this feels like the characters are reclaiming a moral boundary that had collapsed.

Though I also note they are quite a bit more focused on the community element: do they want to be a community together; can they, personally, deal with those requirements; can they find a place and resources to do it; etc.

Other communities exist, but are overpoweringly and explicitly ingroup-eats-outgroup or even ingroup-eats-ingroup in the sense of being exploitative.

Chiefly because this is walking face-first into a race-to-the-bottom condition on purpose. There is a complete lack of causal information here.

I should probably clarify that I don't believe this would be the chain of reasoning among alignment motivated people, but I can totally accept it from people who are alignment-aware-but-not-motivated. For example, this sort of seems like the thinking among people who started OpenAI initially.

A similar chain of reasoning an alignment motivated person might follow is: "3.5 years ago I predicted 5 years based on X and ... (read more)

I agree. I think of this as timelines not being particularly actionable: even in the case of a very short timeline of 5 years, I do not believe that the chain of reasoning would be "3.5 years ago I predicted 5 years, and I also predicted 1.5 years to implement the best current idea, so it is time to implement the best current idea now."

Reasoning directly from the amount of time feels like a self-fulfilling prophecy this way. On the other hand, it feels like the model which generated the amount of time should somehow be strategically relevant. On the other ... (read more)

I upvoted this post - it is a good stab based on the easily accessible public information and a look at relevant theory.

Hypothesis 5 is the path to victory here. The core problem is that (almost) everyone is wrong about (almost) everything, and the least wrong people do not form a group. Some examples:

Moderates do not exist. By this I mean there is not and never was any such group of people. The existence of moderates is a mistake in tabulating the results of political surveys. The mistake looks like this: you might have a survey with multiple responses, a... (read more)

I also thought these looked similar, so I dedicated a half-hour or so of searching and I could not turn up any relation between either of the authors of the Research Gate summary and Boyd or the military as far as their Wikipedia pages and partial publication lists go. It appears those two have been writing books together on this set of principles since 2001, based on work going back to the 60's and drawing from the systems management literature.

I also checked for some links between Rickover and Boyd, which I thought might be valid because one of Boyd's ot... (read more)

I’m pretty sure I got this advice from Yudkowsky at some point, in a post full of writing advice, but I can’t find the reference at the moment.

I think this is in The 5 Second Level, specifically the parts describing and quoting from S. I. Hayakawa's Language in Thought and Action.

I am super curious about how you conceptualize the relationship between the theorist's theory space problem and the experimentalist's high dimensional world in this case. For example:

• Liberally abusing the language of your abstraction posts, is the theory like a map of different info summaries to each other, and the experimentalist produces the info summaries to be mapped, or the theorist points to a blank spot in their map where a summary could fit?
• Or is that too well defined, and it is something more like the theorist only has a list of existing summaries

This is totally different from creating comfort. I think lots of folk get this one confused. Your comfort is none of my business, and vice versa. If I can keep that straight while coming from a same-sided POV, and if you do something similar, then it's easy to argue and listen both in good faith.

I agree that same-sidedness and comfort are totally different things, and I really appreciate the bluntness of same-sidedness as a term. I I also think you are undervaluing comfort here. People who are not comfortable do not reveal their true beliefs; same-sidednes... (read more)

I have just recently been wondering where we stand on the very basic description of the problem criteria for productive conversations. Of late our conversations seem to have more of the flavor of proposal for solution -> criticism of solution, which of course is fine if we have the problem described; but if that were the case why do so many criticisms take the form of disagreements over the nature of the problem?

A very reasonable objection is that there are too many unknowns at work, so people are working on those. But this feels like one meta-problem, ... (read more)