I can corroborate; I also somehow read that as 1M instead of 1.4M the first time.
It is quite strange to me that you would describe this story as "light and pleasant." I can understand funny/absurd for the right sense of humor that is not mine.
With such a disclaimer, I think I would have been less confused but still not positive on the text. Maybe I'd be less negative? Neutral, even? A lack of disclaimer is not really the core of it for me though.
I have strange tastes in fiction, I think, and normally after reading something that anti-resonated with me so much, I'd just stop reading. But, I read to the end because a bunch of upvotes signaled to me that I might otherwise miss out on something later in the text.
I live in the Bay Area and work in tech and follow AI news. For me, I already knew that there are traits I don't... (read more)
As an update, I've now gotten a bunch of useful responses of why people liked this! I was worried people would interpret me as being needlessly negative, but it seems mostly not so. I think I'm still far from enjoying this myself, but I do think I've learned a good amount from the answers I got, and I got some interesting things to think about.
I just went and read that one and found it interesting, yes!
That said, even if I'm unlikely to get a satisfactory response, I still want to ask. I would like to be able to better predict what other people like/think. And sometimes, I've even found that understanding someone else can help find new dimensions to appreciate :)
I personally don't believe "writing quality" can be divorced from content, and if you shove a bunch of words together in a "masterful way" but don't say anything, I don't want to read that and therefore would call that bad writing.
Upon slightly more reflection, I think I can appreciate on an intellectual level the quality of the writing if the goal is to evoke disgust. People (myself included, on occasion) partake in Horror content, which is also traditionally a negative emotion. I haven't heard of a Revulsion genre before, so I didn't really consider that this might be a thing people pursue, but I would still be a little surprised if that was what most people got out of the post.
I would also be surprised if the source of all the upvotes was just that it is """high quality""" writing. I usually find LessWrong to be more focused on content, and I still want to know what other people see in this post.
I did not enjoy this. I did not feel like I got anything out of reading this.
However, this got curated and >500 karma, so I feel like I must be missing something. Can anyone inform me: Why did other people enjoy this? The best theory I can scrounge together is that this is "relatable" in some way to people in SF, like it conveys a vibe they are feeling?
If the goal is to evoke a sensation of disgust with the characters, then I guess you've succeeded for me. I already knew I would not like the sort of person described in this story, though, so I didn't feel like I learned anything as a result, but I could see how something like that could be useful for others. I essentially just felt disgust the entire time reading this.
Artificial General Intelligence, AGI, is an AI, that can do anything an individual human can do, (especially on economically productive metrics).
Artificial Superintelligence, ASI, is an AI, that can do much more than all of humanity working together.
These definitions wouldn't be suitable for a legal purpose, I imagine, in that they lack a "technical" precision. However, in my mind, there is a very big difference between the two, and an observer wouldn't be likely to mislabel a system as ASI when it is actually AGI, or vice versa.
Yet, in my mind, one of the biggest risks of AGI is that it is used to build ASI, which is why I still agree with your post.
I like most of this post, but:
AGI != ASI. Defining AGI, and only post ante-fining a company into oblivion that makes AGI may be enough to prevent the death of humanity. I would put pretty good odds on it being enough, as long as it was strongly enforced and detected.
I would still support regulation preventing AGI, I just want the terminology to be straight. ASI is the thing that IABIED.
Suppose you're a human, and you think we're onto something with modern physics. Humans have a brain that does not exceed a meter in radius. Humans have a brain that weighs less than 100kgs. By the Bekenstein bound, we know the average human brain could have at most 2.6*10^42 bits before forming a black hole, and for the upper bounds I gave, that number is larger, but note, it is still finite.
Now let's say you want to make decisions. What does it mean to make decisions? That's tricky, but we can say it has something to do with performing actions based on sense-perceptions. Let's assume that you are fully aware of your... (read 278 more words →)
Besides cryonics, if you can "revive" someone by figuring out the exact parameters of our universe, simulating history, and thusly finding the people who died and saving them, this changes the calculus further, as now we need to consider extending the lifespan of everyone who ever lived.
Mind uploading, besides directly increasing lifespan, could also result in copies and branches of selves. I find it extremely difficult to reason about the utility of having N copies of myself.
It's unclear to me if this possible, but there's at least uncertainty for me here, when we talk about superintelligence.