This is a special post for quick takes by noma. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
An elephant is probably close to the global optimum for turning plants into elephant protein. This is all evolution has optimized for billions and billions of years. It still takes 15-20 years to fully grow an elephant.
Given this example. Why would we expect ASI to industrially foom soon? The world economy is probably in complexity as great as turning plant matter into elephant tissue. Evolution is an ASI at turning dead matter into animals, and it still takes decades to fully grow the thing.
I am recently thinking that computational irreducibility implies longer timelines for industrial foom. Simply put, everything that is being solved right now are things that are computationally reducible. Coding, math, accountants, lawyers, doctors. Such professions are almost always computationally reducible as any work in a bureaucracy is reducible to some set of rules somewhere. Being an accountant is mostly reducible to a set of principles, computer programs are of course reducible to computer programs, and doctors in their head merely follow very simple pattern matching algorithms.
The rest of the world is not like that. Evolution isn't like that, and the human economy isn't like that either.
Chinese economic growth as a model for the singularity. Some 50-100 years ago some 1.4 billion humans(oids) came online in the world economy. And. Nothing happened. Real gdp world growth stayed constant. At most the meteoric rise of China helped to keep real world economic growth at the same level.
I suspect until there are at least 1.4 billion humanoid robots on earth, we will not see AI show up in the growth statistics. This seems at least 10-20 years away still.
People warning about the risks of AI will have the issue that nowhere in the real economy AI will be seen for a very long time. Prompting eventually politicians and economists to say 'what is all this fuss about'
Could we create AI societies with checks and balances? AI workers do tasks, AI cops (smaller, simpler models) monitor for misbehavior, AI prosecutors assemble evidence, AI defense agents challenge accusations and look for false positives, AI watchdogs inspect the cops and judges and we could have competing institutions using different models/training methods/owners and incentives.
Specifically, I think the AI cops, prosecutors and defense agents can be highly specialized and less intelligent models than the AI workers as they only have to understand this more limited domain of having the workers stay within the bounds of the task. Because these are more specialized and less intelligent, we humans could feel more confident that these are aligned.
The singularity can not happen without China's permission. The reason for this is because China manufactures everything, and the rest of the world can not manufacture anything without China's permission. If you wanted a robotics led (industrial) singularity then China gets a veto over you building those robots in the first place.
That's the gist of the post.
For more info
I did a quick take even though I made a post already because the post itself got too little traction, and the above seems like key knowledge to me that people should be aware of.
Why I prefer virtue ethics over utilitarianism, except for very short projections into the future .
The basic argument is simple: I can judge someone's virtue very quickly and I can't predict the future. With a few observations over a few days I can tell if a person is generally honest or trustworthy, is courageous, epistemically rigorous, etc.
However, I can't predict the future very far out. I can't predict in even 6 months what the world is going to be like. This makes utilitarianism more of a mental abstract exercise with some uses in certain bureaucratic decision making processes rather than something to rely on in your personal life.
Robocurve is working to be the METR of robotics progress, they show that GPT-6 Astra scored 95% on a robot control task, up from Fable 5.1's 40%, with 6.2x fewer output tokens at 2.3x lower cost. Recent LLM releases seem to be able to control robots somewhat competently out of the box without any embodiment gap, and has moved my own timelines for a genuine industrial transformation of the economy forward
https://tweetviewer.com/twitter-viewer/chooi_jeq?tweet=2096064315115839904
What if abortion is murder? I've recently been entertaining religious moral frameworks and taking them seriously as a thought exercise. And then it made me realize thatL "if abortion is murder, I live in a society that genocides millions of babies every year". Huh.
Doesn't sound like a good society to live in. So I now hope that abortion is not murder, and I think it probably isn't. Sort of, but not entirely not either. Perhaps.
If abortion is murder, so is contraception. And abstinence. All of them have the primary impact of preventing a human mind from coming into existence. The small mass of brain cells in a fetus is clearly less conscious, in every sense, than other adult mammals.
Factory farming isn't exactly murder since those lives wouldn't exist without it, but it's definitely torture. And yeah our society does it without too much concern. Not a great look for ole humanity. But we can learn.
I do think it's worth pointing out that the typical view of "pro-life"/"anti-abortion" folks is that a fetus, as an individual organism, is "a life", but reproductive cells like sperm and eggs, on their own, are not. And it is generally the status as an organism that they say grants moral personhood, not brain function or other specific demonstration of abilities.