217
Too Soon
20testingthewaters
13Karl von Wendt
24Kamil
15Karl von Wendt
2Auspicious
2Karl von Wendt
2Auspicious
1OVERmind
11Seth Herd
10JenniferRM
6Gordon Seidoh Worley
4JenniferRM
6Logan Riggs
5Anders Lindström
5jmh
4Haiku
2puccibets
1Gurkenglas
-2Forza
New Comment
I'm really sorry for your loss. It's a crushing thing and as my parents get older I often feel that gnawing terror and anxiety as well. I hope you can find some peace eventually.
1I'm sorry for your loss. I would just like to point out that proceeding cautiously with AGI development does not mean that we'll reach longevity escape velocity much later. Actually, I think if we don't develop AGI at all, the chances for anyone celebrating their 200th birthday are much greater.
To make the necessary breakthroughs in medicine, we don't need a general agent who can also write books or book a flight. Instead, we need highly specialized tool AI like AlphaFold, which in my view is the most valuable AI ever developed, and there's zero chance that it will seek power and become uncontrollable. Of course, tools like AlphaFold can be misused, but the probability of destroying humanity is much lower than with the current race towards AGI that no one knows how to control or align.
1It is my opinion as an aging researcher, for what it is worth, that the chances of living 200 years by anyone currently alive round to 0% if we do not develop AGI. We may get away with not developing strong superintelligence, but I consider the development of AGI a necessity. Knowing this, you may proceed accordingly and do your EV calculations. Maybe it is worth the risk or maybe it is not.
Could you explain why exactly AGI is "a necessity"? What can we do with AGI that we can't do with highly specialized tool AI and one ore more skilled human researchers?
Not the person you're responding to, but my guess is that without general AI, we wouldn't know the right questions to ask or which specialized AIs to create.
Thanks for your comment! If we talk about AGI and define this as "generally as intelligent as a human, but not significantly more intelligent", then by definition it wouldn't be significantly better at figuring out the right questions. Maybe AGI could help with that by enhancing our capacity for searching for the right questions, but it shouldn't be a fundamental difference, especially if we weigh the risk of losing control over AI against it. If we talk about superintelligent AI, it's different, but the risks are even higher (however, it's not easy to draw a clear line between AGI and ASI).
All in all, I would agree that we lose some capabilities to shape our future if we don't develop AGI, but I believe that this is the far better option until we understand how to keep AGI under control or safely and securely align it to our goals and values.
Fair point. I basically agree with that - AGI would give us broader capabilities than narrow AI, but certainly would also carry greater risk.
What about the enhancement of human intelligence that was discussed here? (For example How to make superkids - LessWrong).
They probably have more than a 1% chance of success and could accelerate anti-aging research. Even if you consider the current research situation critically stalled.
I am sorry for your loss. Death is natural but it is so, so bad.
I'm assuming you're posting this here in part to foster a discussion of this tension between preventing deaths in the short term and taking more risks on killing everyone by getting alignment wrong if we rush toward AGI.
This tension is probably going to become more widespread as the concept of AGI becomes more prominent. Many people will want faster progress, in hopes of saving themselves or their loved ones. Longtermism is pretty dominant here on LW, but it is very much a minority view in society at large. Thus, this urge to rush will have to be countered by spreading an awareness of how rushing toward AGI is increasing the odds for older people, while risking the lives of their children and grandchildren. And all of the glorious generations to follow - while most people aren't longtermist, the idea of unimaginable flourishing does hold some weight in their minds, and it isn't that ahrd to imagine in broad form.
I face this dilemma myself. At 50 and in imperfect health, my likely end falls somewhere in the middle of my predicted range of hitting longevity takeoff. Any small speedup or slowdown might shift my odds substantially. I don't know what I'd do if I had real power over our rate of progress, but I don't. So I'll continue advocating that we slow down as much as we can, while also working as fast as we can to align our first AGI/ASIs. That speed will improve our odds of collective survival in the likely case that we can't slow down substantially. And it might even save a few more of the precious unique minds now alive.
3🕯️
My mom died last December, and part of the grief is in how hard it is to say (to people who loved her, and miss her, like I do, but don't have the same awareness of history) what you've said here about your mom, and timelines, and how much potentially fantastic future our mothers missed out on. Thank you for putting some of that part of "that lonely part of the grief" into words.
1Sorry that you also lost your mom. 🫂
A sentiment that didn't quite make it into the piece is that my anger and grief has been transformed into steadfastness by my love for her. The idea for this post came from a sense of determination that her death would mean something to others. That steadfastness has also given new fuel to my other projects. I'm determined to get my book finished in time to influence the course of AI. I'm also determined to live the best life I can, and one worthy of my mom's sense of fun, if we really do only have dozens of months left.
That seems nice. I have not acquired steadfastness (yet (growth mindset?)) but perhaps "find things from which I could justifiably draw steadfastness as a resulting apparent trait" would be a useful tactic to try to apply. I have mostly optimized for flexibility, such as to be able to react to whatever happens, and then be able to nudge everything closer back towards The Form Of The Good... but the practical upshot doesn't look like steadfastness from the outside, I don't think.
Mom would have approved of less "apparent chaos from a distance without the ability to see the causal details" in my life. One of her folksy mantras was "be normal and good" and it was a family joke that my brother and I would always object "we can't do that! look at the world, you have to pick one!"
1In the next two decades we're likely to reach longevity escape velocity: the point at which medicine can increase our healthy lifespans faster than we age.
I have the same belief and have thought about how bad it’d be if my loved ones died too soon.
Sorry for your loss.
1Angry that doctors had spent years teaching her to delay treatment by dismissing her concerns.
Sorry for your loss, but thank you for reminding us how precious life is.
The quoted sentence from your post is I believe the main reasons why doctors (doctors not surgeons) will be one of the first high-status professions to be replaced by AI in a couple of years . If you can just get comprehensive blood work done (you could draw blood in a drop-in booth in a mall and send to a lab) and my some pictures taken of your body and then have a conversation with an AI about the symptoms you experience, the need to go to a psychical grumpy stressed doctor is no more needed or even something you would like to do when you will get much much better and consistent results from the AI doctor who treat you with respect and dignity and do no check his or her watch every minute.
1My sympathies to you and your family. This is one of life's sadness that we all experience at some point. While I don't think this will help easy anything (what could I say to do so?) I have to say when may father died it was too soon -- or felt that way. He lived a long life, died from aging (at nearly 102) and lasted a few days after the pain got too much for him that morphine was regularly administered by the hospice nurse. It was time, and expected. When it happened it was still too soon.
It was easier for me when my mother died about 6 months later but I didn't not have the same sense of loss, being lost, wishing there was more I had done, regrets regarding the relationship and more. And in her case there was even a bit of your experience of an initial mis-assessment (in our case be a sybling) that was the final straw with my mother dying a couple of weeks later.
I hope your father is taking it well (that sounds bad because I know it's nothing but pain, but I cannot think of any better way to say) and has other reasons to view life as worth living rather than feeling that same deep too soon loss and a lack of interest in what life offers.
1I'm sorry for your loss. It is something no one should have to go through.
My father was diagnosed with Parkinson's last year. I have processed and accepted the fact that he is going to die.
Under the circumstances, he is most likely going to die from artificial intelligence at about the same time that I do.
There is no temptation you could give me that would make me risk the end of all things. Not prevention of my father's death. Not the prevention of my death. Not the prevention of my partner's death. I do not need AGI. Humanity as a whole does not need AGI, nor do most people even want it.
Death is horrible, which is why everyone should be strongly advocating for AGI to not be built, until it is safe. By default, it will kill literally everyone.
If you find yourself weighing the lives of everyone on earth and deciding for yourself whether they should be imperiled, then you have learned the wrong lesson from stories of comic book supervillains. It's not our choice to make, and we are about to murder everyone's mothers.
Sorry for your loss.
I believe that if we stop AI research then everyone alive today will be dead by 2150 with near certainty. LEV without AGI does not seem realistic at all…
Death is bad and should go away.
Consider whether working on cryonics does better both on would-have-saved-your-mom and on risks-the-lightcone than working on AI.
I don't understand the argument about the trillion expected lives. Why is it assumed that after the singularity we will have trillions more human children? Why is this not a mind upload and the end of reproduction in its current form? And the people alive at the time of the singularity will simply exist in virtual worlds with AI agents?
I find it strange to ask an ASI to emulate a human so that you can raise in your virtual world. And conditionally 8 billion improved uploaded human minds may not strive to emulate as many variations of human brains as possible in order to fill the universe with them.
1Curated and popular this week
