Cross-posted from The Metasophist. Necessarily speculative. It is commonly argued that artificial general intelligence (AGI), unaligned with human values, represents an existential risk for humanity. For example, in his recent book The Precipice, philosopher Toby Ord argued that there is a 10 percent chance that unaligned artificial general intelligence will...
This is a linkpost for https://www.themetasophist.com/chapter/on-history-has-begun-by-bruno-macaes One of the iconic events during the fall of Hellenic-Classical civilisation was the burning of the library of Alexandria. The conquering Muslims apparently thought that the books therein would “either contradict the Koran, in which case they are heresy, or they will agree with...
Cross-posted from The Metasophist. A key contributor to stagnation could be a growing reluctance to take risk. All creation involves risk, because when embarking on the journey you rarely know the result. As Paul Graham said, you might fear creating something "lame". As society develops, it could become less attractive...
I was surprised by that as well, but I took that from an article by Jules Lobel, Professor of Law, University of Pittsburgh Law School based on a book he wrote:
Influential intellectuals such as Bertrand Russell and the famous physicist Leo Szilard supported preventive war arguments, as did noted mathematicians such as John Williams, a leading figure at the RAND Corporation, and John von Neumann, the founder of gametheory.129 Von Neumann was a particularly strong advocate, remarking in 1950 that “[i]f you say why not bomb them tomorrow, I say why not today?If you say today at 5 o’clock, I say why not one o’clock?”130
For that claim he in turn cites Marc Trachtenberg's History and Strategy, which I do not have access to.
Thanks for the clarification. If that's the plausible scenario for Aligned AGI, then I was drawing a sharper line between Aligned and Unaligned than was warranted. I will edit some part of the text on my website to reflect that.
Thanks for your comment. This is something I should have stated a bit more explicitly.
When I mentioned "single state (or part thereof)", the part thereof was referring to these groups or groups in other countries that are yet to be formed.
I think the chance of government intervention is quite high in the slow take-off scenario. It's quite likely that any group successfully working on AGI will slowly but noticeably start to accumulate a lot of resources. If that cannot be concealed, it will start to attract a lot of attention. I think it is unlikely that the government and state bureaucracy would be content to let such resources accumulate untouched e.g. the... (read more)
Thanks for your comment.
If someone wants to estimate the overall existential risk attached to AGI, then it seems fitting that they would estimate the existential risk attached to the scenarios where we have 1) only unaligned AGI, 2) only aligned AGI, or 3) both. The scenario you portray is a subset of 1). I find it plausible. But most relevant discussion on this forum is devoted to 1) so I wanted to think about 2). If some non-zero probability is attached to 2), that should be a useful exercise.
I thought it was clear I was referring to Aligned AGI in the intro and the section heading. And of course, exploring a scenario doesn't mean I think it is the only scenario that could materialise.
Cross-posted from The Metasophist. Necessarily speculative.
It is commonly argued that artificial general intelligence (AGI), unaligned with human values, represents an existential risk for humanity. For example, in his recent book The Precipice, philosopher Toby Ord argued that there is a 10 percent chance that unaligned artificial general intelligence will result in existential catastrophe for humanity in the coming century.
Less attention is devoted to whether or how aligned AGI could introduce additional existential risk, such as through increasing the probability of conflict. For example, how would one country react to the news that another country is on the verge of developing an AGI fully under their control, a potential geopolitical game changer similar... (read 2194 more words →)
Thanks! There seems to be an openness towards error correction which is admirable and unfortunately uncommon.
I've started browsing and posting here a bit so I should introduce myself.
I've been writing online for around five months and put some draft chapters of a book on my website. The objective is to think about how to immunise a society from decline, which basically means trying to find the right balance between creativity and cohesion (not that they are inversely related—it’s quite possible to have neither). Because I can’t buy into any worldview out there today, I’ve tried to systematise my thoughts into a philosophy I call Metasophism. It’s a work in progress, and most of what I read and write links into that in some way.
Prediction mechanisms are something... (read more)
When it costs 20$ to transport a kg to low-earth orbit we might find a way to to mine palladium that can be sold for $34,115 per kg on earth or gold that can be sold for $60,882 per kg.
It would be interesting to see some kind of analysis of what the effect of asteroid mining could be on the prices of these commodities. For example, the global supply of palladium is just over 200 tonnes, so if asteroid mining could match that the price could fall quite dramatically.
The support provided in the book is purely anecdotal (along the lines of what I discussed above) and doesn't really discuss any other models. The alternative explantions I discuss such as re-religiofication due to material conditions are not mentioned in the book, which is wrote in a somewhat impressionistic manner.
This is a linkpost for https://www.themetasophist.com/chapter/on-history-has-begun-by-bruno-macaes
One of the iconic events during the fall of Hellenic-Classical civilisation was the burning of the library of Alexandria. The conquering Muslims apparently thought that the books therein would “either contradict the Koran, in which case they are heresy, or they will agree with it, so they are superfluous.”
The story may be apocryphal, but it seems the tendency is very real, particularly in the US. Removing authors from reading lists, the knocking of statues, the renaming of buildings – is this a passing fad, or a world-historical event? Just as a nascent Arabic culture once cast off the forms of Hellenism, is a nascent American culture throwing... (read 2784 more words →)
Thanks for elaborating.
I agree with the point about utilities, and the fact that for utility-like services (more specifically, those with overwhelming network effects and economies of scale) it should be illegal to prevent access unless the person to whom service is being denied is doing something illegal.
Cross-posted from The Metasophist.
A key contributor to stagnation could be a growing reluctance to take risk. All creation involves risk, because when embarking on the journey you rarely know the result. As Paul Graham said, you might fear creating something "lame".
As society develops, it could become less attractive to take risk. For example, if you are musically inclined, learning and performing the works of Beethoven, Mozart, Chopin, and Wagner may deliver more instant rewards then creating your own musical forms. There is a large audience for the renowned, little for the unknown. In addition, someone like J.S. Bach could focus more on writing music, because he didn't have any of those four... (read 607 more words →)
It does seem that the Trachtenberg reference basically relies upon individual recollections (which I don't trust), and the following extract from a 1944 letter by Szilard to Vannevar Bush (my bold):
... (read more)