All of MondSemmel's Comments + Replies

What Games These Days?

Board games are always great! I also used to be part of the group that wanted to play games with long play times, but nowadays prefer shorter games.

Anyway, if anyone is looking for more game recommendations, there was a recent LW discussion here.

Recommending Understand, a Game about Discerning the Rules

Thanks for the kind comment! I figured the game might work well as a cooperative experience, so I'm glad to hear that that was indeed the case :).

AGI ruin scenarios are likely (and disjunctive)

I wasn't disputing that Zvi mentioned the blood clot story, I was disputing your characterisation of it. Quoting from literally the first two paragraphs from your link:

And even if all the observed clots were extra, all were caused by the vaccine, all were fatal, and that represented the overall base rate, and we ignore all population-level benefits and economic issues, the vaccine would still be worth using purely for personal health and safety by multiple orders of magnitude.

The WHO and EMA said there was no evidence there was an issue.

This is not consist... (read more)

2tailcalled21d
You are right also said that it was not worth it to deal with even if they were real (which contradicts what the Danish calculations showed, as I mentioned), but the argument he lead with was that they were not real: (I had actually initially misremembered him as leading with the argument that they were not worth dealing with, but when I wrote my original post I decided to go back and double-check and found that he was focusing on them not being real instead. So I rewrote it before posting it to say that he didn't believe them.) What I'm saying is that the Danish government did these sorts of calculations at the time and found the trade-off to be worthwhile; the COVID death rate at the time was extremely low, the restrictions were very loose, and the available alternatice vaccines were plentiful, so avoiding AstraZeneca would according to their calculations have lowish costs, such that it is worth it. Meanwhile, I don't see any calculations by Zvi showing that it Denmark was obviously stupid; it seems like Zvi was mostly just pattern-matching (as did I when I originally shared Zvi's article).
AGI ruin scenarios are likely (and disjunctive)

because he did not believe the blood clots were real

I'd need a source on that. From what I recall, the numbers were small and preliminary but plausibly real, but orders of magnitude below the danger of Covid (which IIRC incidentally also causes blood clots). So one could call the suspension penny-wise but pound-foolish, or some such. Not to mention that IIRC the suspension resulted in a dip in Covid vaccinations, so it's not clear that it was even the right call in retrospect. I also recall hearing the suspension justified as necessary to preserve trust in... (read more)

2tailcalled22d
https://www.lesswrong.com/posts/82kxkCdHeSb8smZZq/covid-3-18-an-expected-quantity-of-blood-clots [https://www.lesswrong.com/posts/82kxkCdHeSb8smZZq/covid-3-18-an-expected-quantity-of-blood-clots] I'm not sure about the specifics and I'm about to go to bed so I can't check. But I think the other countries suspended them after Norway did out of a precautionary principle (i.e. if they are not good for Norway then we should probably re-evaluate whether they are good for us), and then often reinstated them after reevaluating. It seems like a good policy to me for a country to pay attention to findings that a medicine may be surprisingly dangerous. I feel like it would be exactly the kind of mustache-twirling consequentialism that Eliezer calls out as un-genre-savvy to continue using a vaccination that your calculations say aren't worth it in terms of naive cost/benefit because switching out the vaccine policy might harm trust. Like I'd rather not have the government decide that I'm too panicky to allow showing doubt. A dimilar point applies to the suspension in much of Europe. While it might've been consequentially better on some level of analysis for Norway/whoever to take into account what effects their suspension would have on other countries, it feels too manipulative to me, and I'd rather they focus on their job.
AGI ruin scenarios are likely (and disjunctive)

I would appreciate this kind of reply, except I can't update on it if the critique isn't public.

For now, I don't think basic notions like "all governments were incompetent on covid" are particularly easy to dispute?

To provide two examples:

  • The only country I know of to a do human challenge trial for Covid was the UK, and as I understand it, that trial only began in February 2021.
  • I'm not aware of any attempts to ban gain-of-function research, let alone of any bans that were actually implemented.
5tailcalled22d
During COVID, Denmark and a number of other European countries suspended dispensing the AstraZeneca vaccine over worries that it would lead to blood clots. Zvi yelled that all of them were "ludicrously stupid several times over" for doing that, because he did not believe the blood clots were real. However, according to Denmark's official calculations, given the amount of alternative vaccines they had access to, the relative laxness of their COVID restrictions, etc., it was actually worthwhile to use the other vaccines instead of AstraZeneca. The calculations didn't look obviously wrong to me, though I got busy before I could check them properly. As far as I understand, it's currently the general scientific consensus that it can indeed cause blood clots once in a while? These seem like reasonably plausible examples of cases where countries could do better, but given the blood clot mistake I wouldn't want to assume that there aren't problems I've missed.
AGI ruin scenarios are likely (and disjunctive)

Nope, I don't buy it. Having read Zvi's Covid posts and having a sense of how much better policy was possible (and was already being advocated at the time), I just don't buy a framing where government Covid policy can be partly considered as competent. I'm also dubious of treating "the military" as a monolithic black box entity with coherent behavior and goals, rather than consisting of individual decision-makers who follow the same dysfunctional incentives as everyone else.

If you have sources that e.g. corroborate sane military policy during the early Covid months, feel free to provide it, but for now I'm unconvinced.

9tailcalled22d
I don't trust Zvi's COVID posts after sharing one of them with people who were much more clueful than me and getting embarrassed by the resulting critique. However I also don't trust the government response to have been even close to optimal.
1ClipMonger23d
Mossad was allegedly pretty successful procuring large amounts of PPE from hostile countries: https://www.tandfonline.com/doi/full/10.1080/08850607.2020.1783620. [https://www.tandfonline.com/doi/full/10.1080/08850607.2020.1783620.] They also had covert contact tracing, and one way or another their case counts seemed pretty low until Omicron. The first few weeks of COVID lockdowns went extremely well: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7675749/ [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7675749/]
  1. Zvi depends on Twitter for a lot of analysis, which is a really serious Goodhart's law situation, and it resulted in him being really wrong about Goodhart-intensive areas like predicting that the Shanghai lockdown would fail. Again, his analysis is still generally top-notch but it's far from perfect. It's more about outperforming large popular news outlets by as much as possible, which he does plenty of. But international affairs is a different beast entirely when it comes to systematic error.
  2. Economic priorities can be considered more important than public
... (read more)
One-day applied rationality workshop in Berlin Aug 29 (after LWCW)

This might benefit from being cross-posted on the page of Community events, though I don't know if there's a policy against posting paid workshops there.

4ChristianKl18d
I just got online again after 10 days of being offline. I'm doing the organizational work to help to make the workshop happen and now crossposted it to Community events.

Have you searched the EA forum on this topic? Seems like a potentially better resource for questions like this.

2lc1mo
Good idea!
ITT-passing and civility are good; "charity" is bad; steelmanning is niche

I figured, which is why I moderated my statement as only "somewhat" confused :).

ITT-passing and civility are good; "charity" is bad; steelmanning is niche

I am somewhat confused that you provide that comment thread as an example of charity having negative effects, when the thing that spawned that entire thread, or so it seems to me, was insufficient charity / civility / agreeableness (as e.g. evidenced by several negative-karma comments).

2Said Achmiz1mo
It hardly needs saying, but: I do not agree with your assessment.
ITT-passing and civility are good; "charity" is bad; steelmanning is niche

I appreciated this post and found its arguments persuasive. Thanks for writing this!

The one thing I wish had been different was that the essay extensively argues against "argumentative charity", but I never got a sense of what exactly the thing is that's being argued against.

Steelmanning and the Ideological Turing Test get extensive descriptions, while argumentative charity is described as "a complete mess of a concept⁠". Which, fair enough, but if the concept defies definition, I'd instead appreciate a couple examples to understand what not to do.

I figure... (read more)

Open & Welcome Thread - July 2022

I've found a new website bug: copy & pasting bullet points from LW essays into the comment field fails with a weird behavior. I've created a corresponding Github issue.

Toni Kurz and the Insanity of Climbing Mountains

Incidentally, you might get more (reddit) comments if you crosspost this essay on the r/slatestarcodex subreddit. The interests of LW and SSC have some decent overlap, and it's sometimes easier to get comments on reddit than on LW.

Toni Kurz and the Insanity of Climbing Mountains

You're welcome :). Anyway, feel free to delete my typo comments once you've read them; it's not like they serve any further purpose in the comment threads once they're fixed.

Dagger of Detect Evil

Phil held his face inn his hands. -> in his hands

2lsusr1mo
Fixed. Thanks.
Deontological Evil

More typos:

  • That's is ridiculous -> That is ridiculous / That's ridiculous
  • Shouldn't you be infiltrating and destroying good organizations. -> (punctuation) organizations?
  • his enemies arguments -> enemies'
  • That the problem with taking utilitarianism too far. -> That's
2lsusr1mo
Fixed. Thanks.
Open & Welcome Thread - July 2022

Note that Duncan just posted the relevant chapter from the CFAR Handbook as a standalone LW essay.

1mikbp1mo
Cool, thanks. I'll read it!
Toni Kurz and the Insanity of Climbing Mountains

What an intense story! Thanks for writing about it.

If you like stories about huge accidents, you might also enjoy this video episode about a plane accident in 1990 where the captain got sucked outside of the cockpit and the co-pilot had to land the plane alone.

Toni Kurz and the Insanity of Climbing Mountains

Typos:

  • a third of the up the mountain -> of the way up
  • Histerstoisser Traverse -> Hinterstoisser
  • Almen immediately phone Eigergletscher Station -> phoned
  • the head of the mountain rescue committee rules that -> ruled that
  • Finally, willy finished his makeshift guide rope -> Toni finished
  • in case Edi needed to climb down the cliff face -> Toni
  • Are these urges to go forth and conquer, to take great risk for little practical benefit simply evolutionary vestiges? -> benefit, simply
  • of a world since the past -> ? (something like: "of a world long gone")
  • Kleine Sheidegg -> Kleine Scheidegg
1GeneSmith1mo
Thanks. It's a bit embarassing that half the comments on my top-rated post of all time are about typos. I suppose that's what I should expect given I wrote half this story at midnight.
1TAG1mo
Repel -> Rapel (or abseil)
1clearthis1mo
Also: The last name is "Von Almen" not "Almen"
Open & Welcome Thread - July 2022

Also certain types of "unacceptable" speech could be banned by the site. This would stimulate out-of-the-box discussion and brainstorming.

By which mechanism do you expect to improve discussion by introducing censorship?

-2Flaglandbase2mo
I'm completely opposed to any type of censorship whatsoever, but this site might have two restrictions: * Descriptions of disruptive or dangerous new technology that might threaten mankind * Politically or socially controversial speech considered beyond the pale by the majority of members or administrators
Minerva

Question on acronyms: what do SOTA and PaLM mean?

2Algon2mo
State of the art and Pathways Language Model (called PaLM by Google). I edited it to clarify.
Units of Exchange

I have built many PCs over the years (there are at this moment six machines that I built, sitting in this room), and I can tell you that this is not correct.

I've also built a smaller number of PCs over the years, maybe 5-ish. I always set a rough budget, then chose the best components in terms of price/performance (and e.g. silence), relative to that budget. I don't understand how you're ever supposed to get worse performance by using that algorithm while tripling your budget.

To be clear, I'm not suggesting that a random $3000 PC necessarily has higher per... (read more)

4Said Achmiz2mo
Here I will only note that the standard advice in such cases is “buy the cheapest version of whatever it is; use it until it breaks (or no longer satisfies your needs); at that point you will have gotten a great deal of data to inform subsequent purchases, if any”. Doing a great deal of research first, before taking any action whatsoever, is certainly a typical (or perhaps stereotypical) “rationalist” failing, and I do not by any means endorse it. (Indeed you may note that I never said anything about “do[ing] all this effortful research”; there is no particular reason why the sort of thought process to which I alluded in my initial post should take all that long. The difficult part—apparently—is the insight, in the first place, that such an approach is correct.)

Re: the PC build example:

First, it would be foolish to suggest that, in any given category, any more expensive thing is always worse than any less expensive thing, and indeed that is not what I claimed. (Note, again, what I said: “It is a fundamental mistake to think that spending more money necessarily gets you more of anything that you value.”—I did not emphasize ‘necessarily’ in my initial comment, but it’s there for a reason!)

Second, an obvious point, but one whose importance is easy to overlook, is that the PC builds you link to, are not single produc... (read more)

Units of Exchange

My point in that quote was that while these products may be made of e.g. ostensibly better materials, they're inferior relative to your requirements. In your framing (of products that are "just worse", irrespective of requirements), it seems to me like one should be able to buy the $10 toothbrush, sell it for $100, outsell the originally more expensive item, and make $90 profit. As I presume that this doesn't actually happen, I conclude that some customers prefer the product that originally costs $100, and it can ergo not be considered "just worse".

When yo... (read more)

3Said Achmiz2mo
No. They’re not. They’re really, genuinely not. And that’s the mistake you’re making. There is no good reason to believe this to be possible unless you think that purchasing decisions are driven primarily by customers’ accurate evaluations of product quality… which condition happens to be violated precisely in the case where it is commonplace to use price as a signal of quality! Consider the scenario where a customer prefers a product which is, in fact, inferior, given his needs. Does this scenario strike you as incoherent, as described? Or merely impossible in practice? Or neither incoherent nor impossible? If the latter, how common would you say that it is? Unanimous? No, of course not unanimous. There is always the “lizardman quotient”, even among LessWrongers. More generally, whether customers’ judgments are rational given their needs and wants is, indeed, the key piece of this puzzle. People on Less Wrong are not immune to this particular bias (indeed it is my experience that “rationalists” have a huge blind spot when it comes to this topic, due, perhaps, to a greater trust in markets than that of the average person).
Units of Exchange

Many things are like this. It is a fundamental mistake to think that spending more money necessarily gets you more of anything that you value. It is, in fact, quite common for spending more money to get you less quality and less aesthetics and less ease of use and less durability and less reliability and … etc.

When deciding what to buy—which thing, what kind of thing, how many things, etc.—you should not start with consideration of prices of products and then ask how much you’re willing to spend and so on. To do so is to head off in the wrong direction. Yo

... (read more)
4Said Achmiz2mo
What I am saying is that doing as you suggest very often (and, in fact, increasingly often) ends up being completely misleading—not just “not ideal” or some such, but actually much worse than doing as I suggest. But it’s not a valid source of information. That’s my point! I have built many PCs over the years (there are at this moment six machines that I built, sitting in this room), and I can tell you that this is not correct. On this point I agree with you. (But it’s not clear whether this is meant as a counterpoint to anything that I said? It does not seem to be any such thing…)
Units of Exchange

Note that this is the best toothbrush available on the market (given my needs); there are many models that are more expensive, but they are all worse than the $10 model I bought. Let me be very clear about this: if I had spent more money, I would have gotten an inferior productnot in “value per dollar” terms, but in absolute terms.

Here's how I would put this: I like the ISO 9000 definition of "quality" as the "degree to which a set of inherent characteristics fulfils requirements".

When you want to find the best product for yourself, you have some requirem... (read more)

3Said Achmiz2mo
I think that I am not quite getting my meaning across… my point is that, in this case—and in many other cases—are not ‘ostensibly “better” in some Platonic sense’. They’re not better in any practical sense. They’re just worse. They are inferior products. If someone gave one to me as a gift, I would discard it, and buy the $10 toothbrush instead. If the $10 toothbrush cost $100, and the $100 toothbrushes cost $10, without changing any of their other characteristics, I would spend the extra money to get the actually better one. This isn’t about “affordability” at all.
LessWrong Has Agree/Disagree Voting On All New Comment Threads

There's definitely a bug / inconsistency here: the linked comments are in a different order when viewed as a permalink vs. when viewed in the thread itself. But yeah, I was way too quick to assume, based on a single data point, that this was a) a new problem and b) caused by or influenced by agreement karma or the related recent website update. Oops. I thought these things were likely related because, as stated in this thread, only karma (but not agreement) is supposed to dictate how things are ordered; so when I saw a wrong ordering with differing agreeme... (read more)

LessWrong Has Agree/Disagree Voting On All New Comment Threads

Bug: When comment threads are loaded as a permalink, comment sorting is wrong or at least influenced by agreement karma.

Example: This comment thread. In this screenshot, the comment with 2 karma and 1 agreement is sorted above the comment with 8 karma and 0 agreement.

4Ruby2mo
It's possible there's a bug in the comment ordering here that we should look into, but it's very unlikely to be because the agreement voting is being taken into account.
Kurzgesagt – The Last Human (Youtube)

Having skimmed their further reading section for this video, I'm happy to see how seriously this channel takes its research. And as became apparent from the video itself, it was supported by Open Philanthropy and FHI.

Other random things I learned about the channel:

  • The channel is headquartered in Munich.
  • "In 2013 an average video took about 150 hours. In 2015 it was about 250 hours. Now in 2021 we spend around 1200 hours per video." (From their Patreon.)
  • Besides their free "kurzgesagt" videos, they also do commercial animation projects, and it's a bizarre exp
... (read more)
It’s Probably Not Lithium

In German, the tap water is known to be very hard, so essentially no one drinks tap water.

Our local tap water (in a town close to Munich) is roughly as soft as tap water can be, and I drink nothing else.

But if you've found statistics on how countries differ in how much tap water their citizens drink, I'd be interested to see them. Unfortunately, searching for "tap water consumption" includes all the other uses like showering etc.

2gbear6052mo
Hm, this was mostly anecdotal from speaking to German friends (including people in Munich!), so I guess I was speaking too generally. Certainly more people drink bottled water in Germany at restaurants compared to many other countries, but I see that I was overstating the case for at home.
Limits of Bodily Autonomy

I could swear it was frontpaged when I wrote that, but now I'm only 80% sure that it was[1]. Anyway, I figured maybe auto-crossposted posts by high-karma LW posters might automatically get posted as Frontpaged rather than as Personal Blog.

  1. ^

    I welcome evidence both for and against the hypothesis that I hallucinated that.

Pardon the confusion. It was frontpaged, I saw your comment, then moved it back to personal blog. The thought didn't occur to me that you would then be mildly gaslit about your comment!

And no, everything including crossposts get manually processed and frontpaged-or-not. Occasional simple errors make it through. Thx MondSemmel for the comment that pointed this one out.

Limits of Bodily Autonomy

Having politics posts on LW is fine, but they mostly shouldn't be frontpaged and instead remain personal blog posts.

6jefftk2mo
This post is tagged Personal Blog and is not tagged Curated or Frontpage (more [https://www.lesswrong.com/posts/5conQhfa4rgb4SaWx/site-guide-personal-blogposts-vs-frontpage-posts] ), so it's not clear to me what you'd like to be different?
LessWrong Has Agree/Disagree Voting On All New Comment Threads

If there's going to be an agreement-disagreement axis, maybe reconsider how and whether voting strength interacts with it. I saw a comment in this thread which got to -10 agreement from one vote. Which is, if not laughably absurd, certainly utterly unintuitive. What is that even supposed to mean?

LessWrong Has Agree/Disagree Voting On All New Comment Threads

I'm also struggling to interpret cases where karma & agreement diverge, and would also prefer a system that lets me understand how individuals have voted. E.g. Duncan's comment above currently has positive karma but negative agreement, with different numbers of upvotes and agreement votes. There are many potential voting patterns that can have such a result, so it's unclear how to interpret it.

Whereas in Duncan's suggestion, a) all votes contain two bits of information and hence take a stand on something like agreement (so there's never a divergence be... (read more)

Whereas in Duncan's suggestion, a) all votes contain two bits of information and hence take a stand on something like agreement

I didn't notice that! I don't want to have to decide on whether to reward or punish someone every time I figure out whether they said a true or false thing. Seems like it would also severely enhance the problem of "people who say things that most people believe get lots of karma".

LessWrong Has Agree/Disagree Voting On All New Comment Threads

Another option would be heading-based voting, i.e. if you use headings in your comments, each one of those could become votable, or be treated internally as separate comments to vote on and reply to.

However, one problem with all such approaches (besides the big issue of increased UI complexity, of course) is that they're kind of incompatible with the ability to edit one's own comments - what if someone votes on a block quote or heading in you comment, and then you edit that part, or remove it altogether?

LessWrong Has Agree/Disagree Voting On All New Comment Threads

And while I'm already in my noticing-tiny-things perfectionist mode: The line spacings between paragraphs and bulleted lists of various indentation levels seem inconsistent. Though maybe that's good typographical practice?

See this screenshot from desktop Firefox: there seem to be 3+ different line spacings with little consistency. For example:

  • big spacing between an unindented paragraph and a bullet point
  • medium spacing between bullet points of the same indentation level
  • medium spacing between a bullet point of a higher indentation level, followed by one with
... (read more)
LessWrong Has Agree/Disagree Voting On All New Comment Threads

Also, also - it's a bit confusing that karma defaults to a normal upvote by the poster, but the agreement defaults to none (but it can be added by the poster if they actually agree with themselves)?

On this point, I suggest making it so that people cannot vote agree/disagree on their own comments. It's one thing to say "I find my own comment here so valuable that I use a strong upvote on it so more people see it" - that's weird and somewhat discouraged by the community, but at least carries some information.

But what's the equivalent supposed to be for agreement? "I find my own comment so correct that I strongly agree with it"? Just disallow that in the software.

9Kaj_Sotala2mo
Now I'm imagining someone writing a devil's advocate kind of comment they themselves disagree with, and then strong-downvoting agreement.
LessWrong Has Agree/Disagree Voting On All New Comment Threads

Even when it comes to comments, I often wish people would break up their long comments more so I could vote separately on different claims.

3Rob Bensinger2mo
What about a feature where you can mark block quotes in your own comment with 'strong agree', 'weak agree', 'weak disagree', or 'strong disagree'?
1MSRayne2mo
As someone who thinks out loud, I probably would annoy the heck out of you. I regularly make like five or six different orthogonal claims in one comment. My standard is "stream of consciousness, then edit like five times as I think of more things to say / better ways to say them". It's kind of a bad habit though and I should just make more comments. Anyway, my point is that I agree and would like to be able to delineate claims in my comments too.
LessWrong Has Agree/Disagree Voting On All New Comment Threads

Aesthetically speaking, this current implementation still looks rather ugly to me. Specific things I find ugly:

  • Left-right arrows in the comments vs. down-up arrows on LW posts.
  • The visible boundary box around normal votes & agree-disagree votes.
    • I might understand vertical lines between date & normal upvotes, and between normal upvotes & agree-disagree votes. But why do we need boundary lines at the top & bottom?
    • And rather than even vertical lines, maybe just extra whitespace between the various votes might already be enough?
  • The boundary boxe
... (read more)
6MondSemmel2mo
And while I'm already in my noticing-tiny-things perfectionist mode: The line spacings between paragraphs and bulleted lists of various indentation levels seem inconsistent. Though maybe that's good typographical practice? See this [https://i.imgur.com/WOvTWHw.png] screenshot from desktop Firefox: there seem to be 3+ different line spacings with little consistency. For example: * big spacing between an unindented paragraph and a bullet point * medium spacing between bullet points of the same indentation level * medium spacing between a bullet point of a higher indentation level, followed by one with a lower indentation level * tiny spacing between a bullet point of a lower indentation level, followed by one with a higher indentation level * big spacing between the end of a comment and the "Reply" button
LessWrong Has Agree/Disagree Voting On All New Comment Threads

I appreciate this voting system in controversial threads, but find it a bit overkill otherwise.

Maybe you could make this option "enabled by default", so if a thread creator doesn't think it's a good fit for a post, they can opt out of it by unchecking a box?

6tutor vals2mo
Giving a post's creator the option to enable/disable this secondary axis voting seems valuable. A post creator will probably know when his post will generally need nuanced comments with differing opinions, or is more lightweight (ie. what's your favourite icecream) and would appreciate the lighter UI.
The inordinately slow spread of good AGI conversations in ML

The two images in this post don't load for me. As I understand it, they're embedded here from Twitter's content delivery network (CDN), and such embedded images don't always load. To avert problems like this, it's better to copy images in such a way that they're hosted on LW's own CDN instead.

2Rob Bensinger2mo
Thanks, fixed!
2gwillen2mo
It seems to be "responsive"; I wasn't seeing it on desktop either, until I made the window wider.
Announcing the DWATV Discord

Crossposted LW posts list their original source next to the author's username. See this screenshot.

3anonce2mo
Not on mobile, in my experience.
Open & Welcome Thread - June 2022

From Algorithms to Live By, I vaguely recall the multi-armed bandit problem. Maybe that's what you're looking for? Or is that still too closely tied to the explore-exploit paradigm?

4Oscar_Cunningham2mo
I got a good answer here: https://stats.stackexchange.com/q/579642/5751 [https://stats.stackexchange.com/q/579642/5751]
2Oscar_Cunningham2mo
Right. The setup for my problem is the same as the 'bernoulli bandit', but I only care about the information and not the reward. All I see on that page is about exploration-exploitation.
How I Got So Much GHT

dramatically lower my bar for posting on LessWrong. The votes will sort it out. If it's not interesting, people will just ignore it, but sometimes a post that seems not worth writing is actually something people are really excited to read, and people are also happy to read a quickly written post rather than no post at all

Something I just learned is that the audiences and interests of even nominally adjacent communities like LW and e.g. SSC are pretty different. This question post of mine saw no interest on LW, but a (to me) frankly surprising amount of int... (read more)

We will be around in 30 years

And keep in mind that you need to do that without being discovered and in a super short amount of time.

While I expect that this would be the case, I don't consider it a crux. As long as the AGI can keep itself safe, it doesn't particularly matter if it's discovered, as long as it has become powerful enough, and/or distributed enough, that our civilization can no longer stop it. And given our civilization's level of competence, those are low bars to clear.

We will be around in 30 years

What about plans like "hack cryptocurrency for coins worth hundreds of millions of dollars" or "make ransomware attacks" is not trivial? Cybercrimes like these are regularly committed by humans, and so a superintelligence will naturally have a much easier time with them.

If we postulate a superintelligence with nothing but Internet access, it should be many orders of magnitude better at making money in the pure Internet economy (e.g. cybercrime, cryptocurrency, lots of investment stuff, online gambling, prediction markets) than humans are, and some humans already make a lot of money there.

1mukashi2mo
Oh yes, I don't have any issues with a plan where the machine hacks crypto, though I am not sure how capable would be of doing that without raising any alarms from any group in the world, how it could guarantee that someone is not monitoring it. After that, remember you still need a lot of inferential steps to get to a point where you successfully deploy those cryptos into things that can exterminate humans. And keep in mind that you need to do that without being discovered and in a super short amount of time.
We will be around in 30 years

Maybe it reduces the population enough for an AGI to target the rest of us or prevent us from rebuilding, though.

Yeah, I'm familiar with the arguments that neither pandemics nor nuclear war seem likely to be existential risks, i.e. ones that could cause human extinction; but I'd nonetheless expect such events to be damaging enough from the perspective of a nefarious actor trying to prevent resistance.

Ultimately this whole line of reasoning seems superfluous to me - it just seems so obvious that with sufficient cognitive power one can do ridiculous things -... (read more)

We will be around in 30 years

What current defenses do you think we have against nukes or pandemics?

For instance, the lesson from Covid seems to be that a small group of humans is already enough to trigger a pandemic. If one intended to develop an especially lethal pandemic via gain-of-function research, the task already doesn't seem particularly hard for researchers with time and resources, so we'd expect a superintelligence to have a much easier job.

If getting access to nukes via hacking seems too implausible, then maybe it's easier to imagine triggering nuclear war by tricking one n... (read more)

1MichaelStJules2mo
Personal protective equipment and isolation can protect against infectious disease, at the very least. A more deadly and infectious virus than COVID would be taken far more seriously. I think nuclear war is unlikely to wipe out humanity, since there are enough countries that are unlikely targets, and I don't think all of the US would be wiped out anyway. I'm less sure about nuclear winter, but those in the community who've done research on it seem skeptical that it would wipe us out. Maybe it reduces the population enough for an AGI to target the rest of us or prevent us from rebuilding, though. Some posts here: https://forum.effectivealtruism.org/topics/nuclear-warfare-1 [https://forum.effectivealtruism.org/topics/nuclear-warfare-1] https://forum.effectivealtruism.org/topics/nuclear-winter [https://forum.effectivealtruism.org/topics/nuclear-winter]
We will be around in 30 years

Such a policy invites moral hazard, though. If many people followed it, you could farm karma by simply beginning each post with the trite "this is going to get downvoted" thing.

7Conor Sullivan2mo
I think we should have a community norm that commenting on (or whining about) up/downvotes should be a separate post from object-level discussions, or just avoided entirely.
Load More