All of RobertM's Comments + Replies

If you're saying you can't understand why Libertarians think centralization is bad, that IS a crux and trying to understand it would be a potentially useful exercise.

I am not saying that.  Many libertarians think that centralization of power often has bad effects.  But trying to argue with libertarians who are advocating for government regulations because they're worried about AI x-risk by pointing out that government regulation will increase centralization of power w.r.t. AI is a non-sequitur, unless you do a lot more work to demonstrate how the increased centralization of power acts contrariwise the libertarian's goals in this case.

Your argument with Alexandros was what inspired this post, actually.  I was thinking about whether or not to send this to you directly... guess that wasn't necessary.

The question is not whether I can pass their ITT: that particular claim doesn't obviously engage with any cruxes that I or others like me to have, related to x-risk.  That's the only thing that section is describing.

1Logan Zoellner17h
I think maybe you misunderstand the word "crux".  Crux is a point where you and another person disagree.  If you're saying you can't understand why Libertarians think centralization is bad, that IS a crux and trying to understand it would be a potentially useful exercise.

Yeah, that seems like a plausible contributor to that effect.

Edit: though I think this is true even if you ignore "who's calling for regulations" and just look at the relative optimism of various actors in the space, grouped by their politics.

I was going to write, "surely the relevant figure is how much you pay per month, as a percentage of your income", but then I looked at the actual image and it seems like that's what you meant by house price.

Yes, right tails for things that better represent actual value produced in the world, i.e. projects/products/etc.  I'm pretty skeptical of productivity metrics for individual developers like the ones described in that paper, since almost by construction they're incapable of capturing right-tail outcomes, and also fail to capture things like "developer is actually negative value".  I'm open to the idea that remote work has better median performance characteristics, though expect this to be pretty noisy.

On priors I think you should strongly expect in-person co-working to produce much fatter right-tails.  Communication bandwidth is much higher, and that's the primary bottleneck for generating right-tail outcomes.

1bhauth4d
The Baidu study shows slightly longer right tails for individual productivity with remote work, and IIRC others have shown longer tails for remote work as well. Or did you mean right tails for overall project results?

I don't know how I'd evaluate that without specific examples.  But in general, if you think price signals are wrong or "more misleading than not" when it comes to measuring endpoints we actually care about, then I suppose it's coherent to argue that we should ignore price signals.

1bhauth4d
I wouldn't say that "in general" but there are some situations where I do think price signals mean little. For example: * prices of expensive modern art * prices of expensive clothes * valuations of some startups * salaries of many CEOs America today has large income inequalities, but income inequality < wealth inequality < power inequality. One thing the items on the above list have in common is LARPing by the ultra-wealthy.

Because there's a big difference between "has unsavory political stances" and "will actively and successfully optimize for turning the US into a fascist dictatorship", such that "far right or fascist" is very misleading as a descriptor.

I might agree with a more limited claim like "most people in our reference class underestimate the chances of western democracies turning into fascist dictatorships over the next decade". 

I don't think someone reading this post should have >50% odds on >50% of western democracies turning into fascist dictatorships over the next decade or two, no.  I don't see an argument that "fascist dictatorship" is a stable attractor; as others have pointed out, even countries which started out much closer to that endpoint have mostly not ended up there after a couple of decades despite appearing to move in that direction.

Oh, that might just be me having admin permissions, whoops.  I'll double-check what the intended behavior is.

(Update: I just merged a PR that should fix the issue, i.e. make it clear who's comments got deleted. Should be live in about 7 minutes)

You can see who wrote the deleted comment here (and there's also a link this page at the bottom of every post's comment section).  Not sure if we intend to hide the username on the comment itself, will check.

1Ninety-Three4d
One can cross-reference the moderation log with "Deleted by alyssavance, Today at 8:19 AM" to determine who made any particular deleted comment. Since this information is already public, does it make sense to preserve the information directly on the comment, something like "[comment by Czynski deleted]"?
3interstice5d
On the linked page, I see "[anonymous]" for all values of the user field.

I don't actually see very much of an argument presented for the extremely strong headline claim:

This post aims to show that, over the next decade, it is quite likely that most democratic Western countries will become fascist dictatorships - this is not a tail risk, but the most likely overall outcome.

You draw an analogy between the "by induction"/"line go up" AI risk argument, and the increase in far-right political representation in Western democracies over the last couple decades.  But the "by induction"/"line go up" argument for AI risk is not the ... (read more)

3LeBleu5d
I'm confused why you don't expect some other Republican candidate to do it. Have you not paid attention to Gov. DeSantis's actions in Florida? https://www.sltrib.com/opinion/commentary/2023/05/05/commentary-is-ron-desantis-fascist/ [https://www.sltrib.com/opinion/commentary/2023/05/05/commentary-is-ron-desantis-fascist/] I'm not familiar with Nikki Haley, but this article seems to indicate she is at least far right: https://www.newstatesman.com/quickfire/2023/02/nikki-haley-is-extremist-moderates-clothing-donald-trump [https://www.newstatesman.com/quickfire/2023/02/nikki-haley-is-extremist-moderates-clothing-donald-trump] Mike Pence risked his life to oppose Trump's January 6th coup attempt, so even though he is an Christian evangelical Dominionist, and I vehemently disagree with him on policy, I'm going to count him as pro-democracy. I also couldn't easily find any clear point by point evidence that he's a fascist, separate from Trump. Mostly stuff like this which calls him one, but never backs it up: https://www.google.com/amp/s/www.thenation.com/article/politics/mike-pence-gridiron-january-6th/tnamp/ [https://www.google.com/amp/s/www.thenation.com/article/politics/mike-pence-gridiron-january-6th/tnamp/] So out of the 4 people Politico considers contenders for the Republican nomination, 3 are far right or fascist, and the 1 who is partially pro-democracy is considered not likely to win, but might be able to influence who does. https://www.politico.com/interactives/2023/republican-candidates-2024-gop-presidential-hopefuls-list/ [https://www.politico.com/interactives/2023/republican-candidates-2024-gop-presidential-hopefuls-list/]

This seems like a strange reaction. If an alien read this post and believed the claims, wouldn't they think fascism was pretty likely very much on the rise? There's global trends, and there's a bunch of specific examples. Do you agree with that?

Maybe you have some reasons that this prima facie evidence isn't actually strong evidence. What are those reasons?

But the "by induction"/"line go up" argument for AI risk is not the reason one should be worried; one should be worried for specific causal reasons that we expect unaligned ASI to cause extremely bad o

... (read more)

We've thought about something like a "notes to self" feature but don't have anything immediate planned.  In the meantime I'd recommend a 3rd-party solution if bookmarks without notes don't do the thing you need; I've used Evernote in the past but I'm sure there are more options.

1JNS7d
I could do something like that, however it must work on phone, tablet and PC (iOS, android, windows, Linux) I use multiple devices, and anything 3party seems to be bad in such a situation, especially for someone like me who gets sidetracked so easily

Robotic supply chain automation only seems necessary in worlds where it's either surprisingly difficult to get AGI to a sufficiently superhuman level of cognitive ability (such that it can find a much faster route to takeover), worlds where faster/more reliable routes to takeover either don't exist or are inaccessible even to moderately superhuman AGI, or some combination of the two.

At a guess (not having voted on it myself): because most of the model doesn't engage with the parts of the question that those voting consider interesting/relevant, such as the many requirements laid out for "transformative AI" which don't see at all necessary for x-risk.  While this does seem to be targeting OpenPhil's given definition of AGI, they do say in a footnote:

What we’re actually interested in is the potential existential threat posed by advanced AI systems.

While some people do have AI x-risk models that route through ~full automation (or su... (read more)

9Ted Sanders9d
Interesting. When I participated in the AI Adversarial Collaboration Project, a study funded by Open Philanthropy and executed by the Forecasting Research Institute, I got the sense that most folks concerned about AI x-risk mostly believed that AGIs would kill us on their own accord (rather than by accident or as a result of human direction), that AGIs would have self-preservation goals, and therefore AGIs would likely only kill us after solving robotic supply chains (or enslaving/manipulating humans, as I argued as an alternative). Sounds like your perception is that LessWrong folks don't think robotic supply chain automation will be a likely prerequisite to AI x-risk?

You're welcome to host images wherever you like - we automatically mirror all embedded images on Cloudinary, and replace the URLS in the associated image tags when serving the post/comment (though the original image URLs remain in the canonical post/comment for you, if you go to edit it, or something).

2111

Like, I cannot buy chicken for $0.87/lb, I pay about $6.50/lb

I'm sorry, what?  Like, I can in fact go buy boneless chicken thighs for $6.50/lb at Whole Foods in the Bay Area, but that is not what the average consumer is paying.  Prices are in fact more like $1/lb for drumsticks, $1.5/lb for whole birds, $3/lb for boneless thighs/breasts.

2Roko12d
You're correct, that is a mistake. It's $6.50 per kg, I forgot to convert.

No, Eliezer's explicitly clarified that isn't a required component of his model.

1O O5d
Does he? A lot of his arguments hinge on us shortly dying after it appears.

Humans have all the resources, they don’t need internet, computers, or electricity to live or wage war, and are willing to resort to extremely drastic measures when facing a serious threat.

Current human society definitely relies in substantial part on all of the above to function.  I agree that we wouldn't all die if we lost electricity tomorrow (for an extended period of time), but losing a double-digit % of the population seems plausible.

Also, observably, we, as a society, do not resort to sensible measures when dealing with a serious thread (e.g. c... (read more)

2TAG15d
I would say that anyone stating... (EY, of course) ...is assuming exactly that. Particularly given the "shortly".

test leaving an eighteenth comment

test leaving a seventeenth comment

test leaving a sixteenth comment

test leaving a fifteenth comment

test leaving a fourteenth comment

test leaving a thirteenth comment

test leaving an twelfth comment

test leaving an eleventh comment

test leaving a tenth comment

test leaving a ninth comment

test leaving an eighth comment

test leaving a seventh comment

test leaving a sixth comment

test leaving a fifth comment

test leaving a fourth comment

test leaving a third comment on prod!

test leaving a second comment on prod

test leaving a comment on prod

Yeah, I probably should have explicitly clarified that I wasn't going to be citing my sources there.  I agree that the fact that it's costly to do so is a real problem, but Robert Miles points out, some of the difficulty here is insoluble.

It's very strange to me that there isn't a central, accessible "101" version of the argument given how much has been written.

There are several, in fact; but as I mentioned above, none of them will cover all the bases for all possible audiences (and the last one isn't exactly short, either).  Off the top of of my... (read more)

the presentation of the topic is unpersuasive to an intelligent layperson

There is, of course, no single presentation, but many presentations given by many people, targeting many different audiences.  Could some of those presentations be improved?  No doubt.

I agree that the question of how to communicate the problem effectively is difficult and largely unsolved.  I disagree with some of the specific prescriptions (i.e. the call to falsely claim more-modest beliefs to make them more palatable for a certain audience), and the object-level argum... (read more)

5nicholashalden22d
It's very strange to me that there isn't a central, accessible "101" version of the argument given how much has been written. I don't think anyone should make false claims, and this is an uncharitable mischaracterization of what I wrote. I am telling you that, from the outside view, what LW/rationalism gets attention for is the "I am sure we are all going to die", which I don't think is a claim most of its members hold, and this repels the average person because it violates common sense. The object level responses you gave are so minimal and dismissive that I think they highlight the problem. "You're missing the point, no one thinks that anymore." Responses like this turn discussion into an inside-view only affair [https://www.lesswrong.com/posts/ZQG9cwKbct2LtmL3p/evaporative-cooling-of-group-beliefs].  Your status as a LW admin sharpens this point.

Over the years roughly between 2015 and 2020 (though I might be off by a year or two), it seemed to me like numerous AI safety advocates were incredibly rude to LeCun, both online and in private communications.

I'd be interested to see some representative (or, alternatively, egregious) examples of public communications along those lines.  I agree that such behavior is bad (and also counterproductive).

Against them, The conjecture about what protein folding and ribosomes might one have the possibility to do really weak counterargument, based as it is on no empirical or evidentiary reasoning

I'm not sure I've parsed this correctly, but if I have, can I ask what unsupported conjecture you think undergirds this part of the argument?  It's difficult to say what counts as "empirical" or "evidentiary" reasoning in domains where the entire threat model is "powerful stuff we haven't managed to build ourselves yet", given we can be confident that set isn't em... (read more)

1APaleBlueDot1mo
It matters because the original poster isn’t saying we don’t use it to solve real world problems, but rather that real world constraints (I.e. laws of physics) will limit its speed of advancement. An AI likely cannot easily predict a chaotic system unless it can simulate reality at a high fidelity. I guess Op is assuming the TAI won’t have this capability, so even if we do solve real world problems with AI, it is still limited by real world experimentation requirements.

By contrast, some lines of research where I’ve seen compelling critiques (and haven’t seen compelling defences) of their core intuitions, and therefore don't recommend to people:

  1. Cooperative inverse reinforcement learning (the direction that Stuart Russell defends in his book Human Compatible); critiques here and here.
  2. John Wentworth’s work on natural abstractions; exposition and critique here, and another here.

The first critique of natural abstractions says:

Concluding thoughts on relevance to alignment: While we’ve made critical rem

... (read more)
4Richard_Ngo1mo
Good point. Will edit.

Some combination of:

  • tech debt from design decisions which made sense when rebooting as LW 2.0, but have become increasingly unwieldy as development's continued
    • strictly speaking there were options here that weren't moving off of mongo, but it'd be much more difficult to make sane design choices with mongo involved.  the fact that it can do a thing doesn't mean it does that thing well.
    • mongo's indexing in particular is quite bad, both in terms of how finnicky mongo is about whether it can figure out how to use an index for a given query... and then also
... (read more)

There seems to be much more diversity in human cognitive performance than there is in human-brain-energy-efficiency; whether this is due to larger differences in the underlying software (to the extent that this is meaningfully commensurable with differences in hardware) or because smaller differences in that domain result in much larger differences in observable outputs, or both, none of that really takes away from the fact that brain software does not seem to be anywhere near the relevant efficiency frontier, especially since many trade-offs which were operative at an evolutionary scale simply aren't when it comes to software.

9jacob_cannell2mo
Human mind software evolves at cultural speeds so its recent age isn't comparably relevant. Diversity in human cognitive capabilities results from the combined oft multiplicative effects of brain hardware differences compounding unique training datasets/experiences. Its well known in DL that you can't really compare systems trained on vary different datasets, but that is always the case with humans.

I've heard people be somewhat optimistic about this AI guideline from China. They think that this means Beijing is willing to participate in an AI disarmament treaty due to concerns over AI risk.

I'm curious where you've seen this.  My impression from reading the takes of people working on the governance side of things is that this is mostly being interpreted as a positive sign because it (hopefully) relaxes race dynamics in the US. "Oh, look, we don't even need to try all that hard, no need to rush to the finish line."  I haven't seen anyone seri... (read more)

I've only seen vaguely positive vibes from people who needed Google Translate in order to understand it, like this post from Zvi. He comes to the conclusion that "All of that points, once again, to an eager partner in a negotiation." This isn't obvious at all. Again, a willingness to regulate nuclear power is not a strong signal for a willingness to participate in nuclear disarmament treaty - all states will eventually have some level of nuclear regulation.

which implies by association that brain software is much more efficient as it was produced by exactly the same evolutionary process which he now admits produced fully optimized conventional computational elements over the same time frame, etc

 

I don't believe this would follow; we actually have much stronger evidence that ought to screen off that sort of prior - simply the relatively large differences in human cognitive abilities.

2jacob_cannell2mo
Evolution optimizes population distributions with multiple equilibria and niches; large diversity in many traits are expected especially for highly successful species. Furthermore what current civilization considers to be useful cognitive abilities often have costs - namely in longer neotany training periods - which don't always pay off vs quicker to breeding strategies.

My guess is the interpretability team is under a lot of pressure to produce insights that would help the rest of the org with capabilities work

I would be somewhat surprised if this was true, assuming you mean a strong form of this claim (i.e. operationalizing "help with capabilities work" as relying predominantly on 1st-order effects of technical insights, rather than something like "help with capabilities work by making it easier to recruit people", and "pressure" as something like top-down prioritization of research directions, or setting KPIs which rely... (read more)

3habryka2mo
The pressure here looks more like "I want to produce work that the people around me are excited about, and the kind of thing they are most excited about is stuff that is pretty directly connected to improving capabilities", whereby I include "getting AIs to perform a wider range of economically useful tasks" as "improving capabilities". I definitely don't think this is the only pressure the team is under! There are lots of pressures that are acting on them, and my current guess is that it's not the primary pressure, but I would be surprised if it isn't quite substantial. 

Complexity of value is part of why value is fragile.

(Separately, I don't think current human efforts to "figure out" human values have been anywhere near adequate, though I think this is mostly a function of philosophy being what it is.  People with better epistemology seem to make wildly more progress in figuring out human values compared to their contemporaries.)

4Matthew Barnett2mo
I thought complexity of value was a separate thesis from the idea that value is fragile. For example they're listed as separate theses in this post [https://intelligence.org/2013/05/05/five-theses-two-lemmas-and-a-couple-of-strategic-implications/]. It's possible that complexity of value was always merely a sub-thesis of fragility of value, but I don't think that's a natural interpretation of the facts. I think the simplest explanation, consistent with my experience reading MIRI blog posts from before 2018, is that MIRI people just genuinely thought it would be hard to learn and reflect back the human utility function, at the level that GPT-4 can right now. (And again, I'm not claiming they thought that was the whole problem. My thesis is quite narrow and subtle here.)

That does indeed seem like some progress, though note that it does not really let us answer questions like "what algorithm is this NN performing that lets it do whatever it's doing", to a degree of understanding sufficient to implement that algorithm directly (or even a simpler, approximated version, which is still meaningfully better than what the previous state-of-the-art was, if restricted to "hand-written code" rather than an ML model).

1Logan Zoellner2mo
I think that to the extent we need to answer "what algorithm?" style questions, we will do it with techniques like this one [https://twitter.com/_akhaliq/status/1648856524012371968] where we just have the AI write code.  But I don't think "what algorithm?" is a meaningful question to ask regarding "Modern Disney Style", the question is too abstract to have a clean-cut definition in terms of human-readable code. It's sufficient that we can define and use it given a handful of exemplars in a way that intuitively agrees with humans perception of what those words should mean.
Load More