countingtoten

Posts

Sorted by New

Comments

Covid 11/12: The Winds of Winter

You're not exactly wrong, but OP does tell us people are being irrational in ways that you could use to get cold hard cash.

Has Eliezer ever retracted his statements about weight loss?

People who have more difficulty than most - like me* - in losing weight constitute about 20% of the community. The hard cases are quite rare.

What? What community? My impression is that, if people try to lose weight, the expected result is for them to become more unhealthy due to weight fluctuation. It's not that they'll "have difficulty losing weight," rather their weight will go up and down in a way that harms their health or their expected lifespans. I thought I read statements by Scott - an actual doctor, we think - supporting this. Are you actually disputing the premise? If so, where'd you get the data?

Somerville Mask Usage

This seems like vastly more people masked than I've observed in a crowded suburb. I can't give precise numbers, but it feels like less than a third wearing them, despite distancing being very difficult if you walk in some areas.

Likelihood of hyperexistential catastrophe from a bug?

No. First, people thinking of creating an AGI from scratch (i.e., one comparable to the sort of AI you're imagining) have already warned against this exact issue and talked about measures to prevent a simple change of one bit from having any effect. (It's the problem you don't spot that'll kill you.)

Second, GPT-2 is not near-perfect. It does pretty well at a job it was never intended to do, but if we ignore that context it seems pretty flawed. Naturally, its output was nowhere near maximally bad. The program did indeed have a silly flaw, but I assume that's because it's more of a silly experiment than a model for AGI. Indeed, if I try to imagine making GPT-N dangerous, I come up with the idea of an artificial programmer that uses vaguely similar principles to auto-complete programs and could thus self-improve. Reversing the sign of its reward function would then make it produce garbage code or non-code, rendering it mostly harmless.

Again, it's the subtle flaw you don't spot in GPT-N that could produce an AI capable of killing you.

Likelihood of hyperexistential catastrophe from a bug?

The only way I can see this happening with non-negligible probability is if we create AGI along more human lines - e.g, uploaded brains which evolve through a harsh selection process that wouldn't be aligned with human values. In that scenario, it may be near certain. Nothing is closer to a mind design capable of torturing humans than another human mind - we do that all the time today.

As others point out, though, the idea of a sign being flipped in an explicit utility function is one that people understand and are already looking for. More than that, it would only produce minimal human-utility if the AI had a correct description of human utility. Otherwise, it would just use us for fuel and building material. The optimization part also has to work well enough. Everything about the AGI, loosely speaking, has to be near-perfect except for that one bit. This naively suggests a probability near zero. I can't imagine a counter-scenario clearly enough to make me change this estimate, if you don't count the previous paragraph.

Studies On Slack

This shows why I don't trust the categories. The ability to let talented people go in whatever direction seems best will almost always be felt as freedom from pressure.

How effective are tulpas?

One, the idea was to pick a fictional character I preferred, but could not easily come to believe in. (So not Taylolth, may death come swiftly to her enemies.) Two, I wanted to spend zero effort imagining what this character might say or do. I had the ability to picture Kermit.

How effective are tulpas?

I meant that as a caution - though it is indeed fictional evidence, and my lite version IRL seems encouraging.

I really think you'll be fine taking it slow. Still, if you have possible risk factors, I would:

  • Make sure you have the ability to speak with a medical professional on fairly short notice.
  • Remind yourself that you are always in charge inside your own head. People who might know tell me that hearing this makes you safer. It may be a self-proving statement.
How effective are tulpas?

Sy, is that you?

I started talking to Kermit the Frog, off and on, many months ago. I had this idea after seeing an article by an ex-Christian who appeared never to have made predictions about her life using a truly theistic model, but who nevertheless missed the benefits she recalls getting from her talks with Jesus. Result: Kermit has definitely comforted me once or twice (without the need for 'belief') and may have helped me to remember useful data/techniques I already knew, but mostly nothing much happens.

Now, as an occasional lucid dreamer who once decided to make himself afraid in a dream, I tend not to do anything that I think is that dumb. I have not devoted much extra effort or time to modelling Kermit the Frog. However, my lazy experiment has definitely yielded positive results. Perhaps you could try your own limited experiment first?

Criticism as Entertainment

Pitch Meeting is at least bringing up actual problems with Game of Thrones season 8. But I dare you to tell if early Game of Thrones was better or worse than season 8, based on the Pitch Meeting.

That's gonna be super easy, barely an inconvenience. The video for S8 not only feels more critical than usual, it gives specific examples of negative changes from previous seasons, plus a causal explanation (albeit partial) that the video keeps coming back to. One might even say the characters harp on their explanation for S8 being worse than early seasons.

Also, looking at the PM video on GOT up to that point, the chief criticism I find (and this once it seems like pretty explicit criticism of the art in question) seems to be about the show getting old and also worse as it goes on. Actual quote:

Well, eventually we're going to start running out of main characters, in later seasons, so they're all going to develop pretty thick plot armor.

Of course, whether that makes the early seasons better or whether their failures took a while to become obvious is a difficult question. PM seems to suggest the former, given their fairly blatant assertion that S8 is worse due to being inexplicably rushed. However, that is technically a different question. It leaves open the possibility that the showrunners' ultimate failure was determined by their approach at the start, as the earlier video suggested.

Load More