countingtoten

Posts

Sorted by New

Comments

Covid 5/27: The Final Countdown

How likely is it that a research lab finds a new bat coronavirus and then before publishing anything about it, decides that it's the perfect testbed for dramatic gain of function research?

In China? We're talking about a virus based on RNA, which mutates more easily, making it harder to control. China's government craves control to the point of trying to censor all mentions of Winnie the Pooh, possibly because a loss of control could mean the guillotine.

Covid 1/7: The Fire of a Thousand Suns

And you also want, in that last tweet, to put (at least some of) the burden on these random institutions to then allocate the vaccine based on who has suffered disproportionately? While also obeying all the official restrictions or else, and also when did that become a priority?

You know that we decide which groups are at risk largely by looking at how they've died or suffered so far. Presumably she's hoping we protect people who otherwise seem likely to die, since her remarks seem sensible given her premises.

Covid 11/12: The Winds of Winter

You're not exactly wrong, but OP does tell us people are being irrational in ways that you could use to get cold hard cash.

Has Eliezer ever retracted his statements about weight loss?

People who have more difficulty than most - like me* - in losing weight constitute about 20% of the community. The hard cases are quite rare.

What? What community? My impression is that, if people try to lose weight, the expected result is for them to become more unhealthy due to weight fluctuation. It's not that they'll "have difficulty losing weight," rather their weight will go up and down in a way that harms their health or their expected lifespans. I thought I read statements by Scott - an actual doctor, we think - supporting this. Are you actually disputing the premise? If so, where'd you get the data?

Somerville Mask Usage

This seems like vastly more people masked than I've observed in a crowded suburb. I can't give precise numbers, but it feels like less than a third wearing them, despite distancing being very difficult if you walk in some areas.

Likelihood of hyperexistential catastrophe from a bug?

No. First, people thinking of creating an AGI from scratch (i.e., one comparable to the sort of AI you're imagining) have already warned against this exact issue and talked about measures to prevent a simple change of one bit from having any effect. (It's the problem you don't spot that'll kill you.)

Second, GPT-2 is not near-perfect. It does pretty well at a job it was never intended to do, but if we ignore that context it seems pretty flawed. Naturally, its output was nowhere near maximally bad. The program did indeed have a silly flaw, but I assume that's because it's more of a silly experiment than a model for AGI. Indeed, if I try to imagine making GPT-N dangerous, I come up with the idea of an artificial programmer that uses vaguely similar principles to auto-complete programs and could thus self-improve. Reversing the sign of its reward function would then make it produce garbage code or non-code, rendering it mostly harmless.

Again, it's the subtle flaw you don't spot in GPT-N that could produce an AI capable of killing you.

Likelihood of hyperexistential catastrophe from a bug?

The only way I can see this happening with non-negligible probability is if we create AGI along more human lines - e.g, uploaded brains which evolve through a harsh selection process that wouldn't be aligned with human values. In that scenario, it may be near certain. Nothing is closer to a mind design capable of torturing humans than another human mind - we do that all the time today.

As others point out, though, the idea of a sign being flipped in an explicit utility function is one that people understand and are already looking for. More than that, it would only produce minimal human-utility if the AI had a correct description of human utility. Otherwise, it would just use us for fuel and building material. The optimization part also has to work well enough. Everything about the AGI, loosely speaking, has to be near-perfect except for that one bit. This naively suggests a probability near zero. I can't imagine a counter-scenario clearly enough to make me change this estimate, if you don't count the previous paragraph.

Studies On Slack

This shows why I don't trust the categories. The ability to let talented people go in whatever direction seems best will almost always be felt as freedom from pressure.

How effective are tulpas?

One, the idea was to pick a fictional character I preferred, but could not easily come to believe in. (So not Taylolth, may death come swiftly to her enemies.) Two, I wanted to spend zero effort imagining what this character might say or do. I had the ability to picture Kermit.

How effective are tulpas?

I meant that as a caution - though it is indeed fictional evidence, and my lite version IRL seems encouraging.

I really think you'll be fine taking it slow. Still, if you have possible risk factors, I would:

  • Make sure you have the ability to speak with a medical professional on fairly short notice.
  • Remind yourself that you are always in charge inside your own head. People who might know tell me that hearing this makes you safer. It may be a self-proving statement.
Load More