User Comment Replies

Meta comment: Would someone mind explaining to me why this question is being received poorly (negative karma right now)? It seemed like a very honest question, and while the answer may be obvious to some, I doubt it was to Sergio. Ic's response was definitely unnecessarily aggressive/rude, and it appears that most people would agree with me there. But many people also downvoted the question itself, too, and that doesn't make sense to me; shouldn't questions like these be encouraged?

4Dagon2d

I didn't downvote because it was already negative and I didn't feel the need to pile on. But if it'd been positive, I would have. It's probably an honest question, but it doesn't contain any analysis or hooks to a direction of inquiry. It doesn't explain why investing in google at the retail level is likely to have any impact on speed or alignment of AGI, nor why the stock will do particularly better than already priced in on any given timeframe based on this deal.

4lc2d

My guess is that the question is being received poorly because either: * People agree with me that supporting investment in Google stock because they're going to build "profitably" world-ending AGI is immoral, and downvoted me because of my aggressive+rude posture. * OP is disregarding the efficient market hypothesis on a company with 1MMM market cap for no good reason.

Convince me that humanity *isn’t* doomed by AGI

starship0061mo7

I don't know what to think of your first three points but it seems like your fourth point is your weakest by far. As opposed to not needing to, our 'not taking every atom on earth to make serotonin machines' seems to be a combination of:

our inability to do so
our value systems which make us value human and non-human life forms.

Superintelligent agents would not only have the ability to create plans to utilize every atom to their benefit, but they likely would have different value systems. In the case of the traditional paperclip optimizer, it certainly would not hesitate to kill off all life in its pursuit of optimization.

3astridain1mo

I agree the point as presented by OP is weak, but I think there is a stronger version of this argument to be made. I feel like there are a lot of world-states where A.I. is badly-aligned but non-murderous simply because it's not particularly useful to it to kill all humans. Paperclip-machine is a specific kind of alignment failure; I don't think it's hard to generate utility functions orthogonal to human concerns that don't actually require the destruction of humanity to implement. The scenario I've been thinking the most about lately, is an A.I. that learns how to "wirehead itself" by spoofing its own reward function during training, and whose goal is just to continue to do that indefinitely. But more generally, the "you are made of atoms and these atoms could be used for something else" cliché is based on an assumption that the misaligned A.I.'s faulty utility function is going to involve maximizing number of atoms arranged in a particular way, which I don't think is obvious at all. Very possible, don't get me wrong, but not a given. Of course, even an A.I. with no "primary" interest in altering the outside world is still dangerous, because if it estimates that we might try to turn it off, it might expend energy now on acting in the real world to secure its valuable self-wireheading peace later. But that whole "it doesn't want us to notice it's useless and press the off-button" class of A.I.-decides-to-destroy-humanity scenarios is predicated on us having the ability to turn off the A.I. in the first place. (I don't think I need to elaborate on the fact that there are a lot of ways for a superintelligence to ensure its continued existence other than planetary genocide — after all, it's already a premise of most A.I. doom discussion that we couldn't turn an A.I. off again even if we do notice it's going "wrong".)

Don't die with dignity; instead play to your outs

starship0061mo8

I like this framing so, so much more. Thank you for putting some feelings I vaguely sensed, but didn't quite grasp yet, into concrete terms.

March 2022 Welcome & Open Thread

starship0062mo3

Hello, does anyone happen to know any good resources related to improving/practicing public speaking? I'm looking for something that will help me enunciate better/ mumble less/ fluctuate tone better. A lot of stuff I see online appears to be very superficial.

Russia has Invaded Ukraine

starship0063mo3

I'm not very well-versed in history so I would appreciate some thoughts from people here who may know more than I. Two questions:

While it seems to be the general consensus that Putin's invasion is largely founded on his 'unfair' desire to reestablish the glory of the Soviet Union, a few people I know argue that much of this invasion is more the consequence of other nations' failures. Primarily, they focus on Ukraine's failure to respect the Minsk agreements, and NATO's expansion eastwards despite their implications/direct statements (not sure which one, I'

... (read more)

2Константин Токмаков3mo

I add that the precedent for Russia's actions in eastern Ukraine and Crimea was called the independence and international recognition of Kosovo. Kind of. "Why can they, but we can't?"

4maximkazhenkov3mo

1. Failure? Putin can't win hearts and minds, so NATO is to blame for not delivering those things to him on a silver platter? 2. Damn straight the US is being hypocritical. No one cares any more, years of Russian whataboutism-propaganda has seen to that.

9Vaniver3mo

My understanding is that many of these talking points are unfairly slanted in Russia's favor, and that the situation seems manufactured by the Russian government in order to justify an invasion. [For example, the breakaway republics to the east are in regions where opinion polls are not in favor of secession from the Ukraine, but fighting has been ongoing for years in part because of Russian support of the separatist groups.] My sense of the situation is that, given Russia thinks it could win a war, what treaty could have been offered that would seem superior to them? [Especially given that part of the benefit of fighting the war is the practice for future wars.] The American Government says lots of hypocritical things about regime change and interfering with elections and so on; I think this is bad and wish they wouldn't do it.

This Year I Tried To Teach Myself Math. How Did It Go?

starship0065mo2

I really admire your patience to re-learn math entirely from the extremely fundamental levels on-wards. I've had a similar situation with Computer Science for the longest time where I would have a large breadth of understanding of Comp Sci topics, but I didn't feel as if I had a deep, intuitive understanding of all the topics and how they related to each other. All the online courses I found online seemed disjunct and separate from each other, and I would often start them and stop halfway through when I felt as if they were going nowhere. It's even worse w... (read more)

A non-magical explanation of Jeffrey Epstein

starship0065mo21

Woah.... I don't know what exactly I was expecting to get out of this article, but I thoroughly enjoyed it! Would love to see the possible sequence you mentioned come to life.

App and book recommendations for people who want to be happier and more productive

starship0066mo12

Awesome recommendations, I really appreciated them (especially the one on game theory, that was a lot of fun to play through). I would like to also suggest Replacing Guilt series by Nate Soares for those who haven't seen it on his blog or on the EA forum, a fantastic series that I would highly recommend people to check out.

2matto6mo

Replacing Guilt is also available as a paper book through Amazon. There are times when I wanted to share this series with a friend, but knew immediately that reading and clicking through a series of posts is not something they could do, so having this on a different medium is a game changer for me.

Petrov Day 2021: Mutually Assured Destruction?

starship0068mo31

Attention LessWrong - I do not have any sort of power as I do not have a code. I also do not know anybody who has the code.

I would like to say, though, that I had a very good apple pie last night.

That’s about it. Have a great Petrov day :)

Internal Double Crux

starship0068mo3

Wow! Maybe since I'm less experienced at this sort of stuff, I'm more blown away about this than the average LessWrong browser, but I seriously believe this deserves some more upvotes. Just tried it out on something small and was pleased to see the results. Thank you for this :)

LESSWRONG
LW

All of starship006's Comments + Replies