...contrary to the misleading leaderboard (which their technical paper implies should actually list humans at ~53%, as explained below): The 98% listed as the "Human Panel" score for ARC-AGI-1 is relatively easy to interpret. It was the score of an actual human who attempted all 100 private evaluation tasks.[1] The...
The economist Wassily Leontief, writing in 1966, used the then-recent decline of horses to make vivid what he foresaw as the coming impact of technological advances on workers. In 1910, the horse had remained indispensable for farming, transportation, and even war, despite decades of the radical technological progress of the...
...contrary to the misleading leaderboard (which their technical paper implies should actually list humans at ~53%, as explained below):
The 98% listed as the "Human Panel" score for ARC-AGI-1 is relatively easy to interpret. It was the score of an actual human who attempted all 100 private evaluation tasks.[1] The higher human score of 100% listed for ARC-AGI-2 suggests that the newer benchmark is slightly easier for humans. And the ARC-AGI-2 announcement does nothing to discourage that impression, asserting that it maintains "the same relative ease for humans."[2] An attached technical report, however, explains that ARC-AGI-2 is designed to be more difficult for humans as well as AI systems.[3]
It turns out that the 100% listed... (read 309 more words →)
The economist Wassily Leontief, writing in 1966, used the then-recent decline of horses to make vivid what he foresaw as the coming impact of technological advances on workers. In 1910, the horse had remained indispensable for farming, transportation, and even war, despite decades of the radical technological progress of the Second Industrial Revolution. By 1960, though, horses were obsolete for all of the above, the US horse population having fallen by 85% as a result.
In 1982, when unemployment topped 10%, continuing an apparent "chronic increase in unemployment from one oscillation of the business cycle to the next,"[1] Leontief's warning of a similar fate for humans seemed on track. But the chronic increase ended there,... (read 1379 more words →)
Agreed on the big picture, but I was somewhat surprised to see top models struggling with River Crossing (for which the output length limit has less bite). I was able to solve N=3 River Crossing by hand, though it took 10+ minutes and I misinterpreted the constraint initially (making it easier by allowing a boat rider to "stay in the boat" rather than fully unloading onto the shore after each trip). But in a couple attempts each, Opus 4 and Gemini 2.5 Pro were not able to solve it without web access or tool use. Dropping the temperature to zero (or 0.25) did not help Gemini.
It may be a "the doctor is... (read more)
Money is fungible. It’s kind of stupid that we have an ‘income tax rate’ and then a ‘medicare tax’ on top of it that we pretend isn’t part of the income tax. And it’s a nice little fiction that payroll taxes pay for social security benefits. Yes, technically this could make the Social Security fund ‘insolvent’ or whatever, but then you ignore that and write the checks anyway and nothing happens.
No, Yglesias's point is not invalidated by the fungibility of money. (It's generally a good idea to think twice before concluding that an economics-y writer is making such a basic mistake.) Payroll taxes make up 35% of all federal revenue. The point... (read more)
The good news is that the societies described here are vastly wealthier. So if humans are still able to coordinate to distribute the surplus, it should be fine to not be productively employed, even if to justify redistribution we implement something dumb...
I'm increasingly skeptical that there will be much redistribution to speak of in such a scenario. The vast numbers of people living on $2 a day currently might have something to say about that. What is the historical precedent for a group of humans having as little leverage as even U.S. ex-workers will have in this 99% automation scenario and yet being gifted a UBI, much less a UHI?
I thought I was just speculating about the potential for multiple valid solutions, but now I see that the ARC 2 launch post not only acknowledges the possibility but says ambiguity is sometimes even there by design!
Like ARC-AGI-1, ARC-AGI-2 uses a
pass@2measurement system to account for the fact that certain tasks have explicit ambiguity and require two guesses to disambiguate. As well as to catch any unintentional ambiguity or mistakes in the dataset. Given controlled human testing with ARC-AGI-2, we are more confident in the task quality compared to ARC-AGI-1. [emphasis added]
I continue to question whether 2 out of 9 solve rates by their human testers should have given them such... (read more)
Why is the human baseline so low? This is more tentative, but I'm thinking in terms of two basic possibilities. The reason only 2-5 out of 9 humans found the intended solution could be that it's a well-posed but difficult problem, or it could be that there are 2-3 equally valid solutions and landing on the intended "correct" answer requires pure luck.
Starting with the latter possibility, a non-trivial proportion of these tasks may be ill-posed problems with multiple legitimate answers. Even an ideal solver would still only average 50% (or 33% or lower) on such tasks. If this were the case for all eval tasks, ~50% would not only be the human... (read more)
Here's Olmstead and Rhode:
... (read more)The early gasoline tractors of the 1900s were behemoths, patterned after the giant steam plows that preceded them. They were useful for plowing, harrowing, and belt work but not for cultivating fields of growing crops nor powering farm equipment in tow. Innovative efforts between 1910 and 1940 vastly improved the machine's versatility and reduced its size, making it suited to a wider range of farms and tasks. ...
...the revolutionary McCormick-Deering Farmall (1924) was the first general-purpose tractor capable of cultivating amongst growing row crops. The latter machine was also among the first to incorporate a power-takeoff, enabling it to transfer power directly to implements under tow. A host of
Why do you say this scenario is "without dignity"?
I was surprised to not see much consideration, either here or in the original GD and IC essays, of the brute force approach of "ban development of certain forms of AI," such as Anthony Aguirre proposes. Is that more (a) because it would be too difficult to enforce such a ban or (b) because those forms of AI are considered net positive despite the risk of human disempowerment?
From the Amish perspective, is the broader US society something of an aligned superintelligence? Maybe "super" is too strong, but I think of the Amish position as a model for how I hope ASI treats humans.
Thinking out loud, the thing is that there are analogs to the non-feed upkeep costs (shelter and farmer/veterinarian labor) for humans. Though some work, like composing poems, requires little more than physical sustenance to be performed well, most human production requires complementary inputs, principally various equipment or machinery. The question then comes down to whether you want to invest in such human-augmenting equipment as opposed to a fully automated solution.
For example, suppose total production is . Then optimal capital allocation means so that human-augmenting capital falls as robot productivity increases relative to human-in-the-loop productivity. Then the real wage, the marginal product of labor, is proportional to . If both and grow exponentially, the condition for the... (read more)