Consumers Sue Anthropic, OpenAI, SpaceXAI and Google Over Alleged AI Pact
I hope that the judges are sane enough to prevent this from punishing AI labs for focusing on safety...
by buying a subscription to a service, you are subscribing not only to the service that exists today, but to a stream of improvements to that service, including improvements that have not yet been announced or developed, delivered as fast as possible
Is this as crazy as it sounds to me, or are there some legal precedents that would support such perspective?
What is one to do when someone genuinely says "I know my views are contradictory, but...", "I'm never going to accept that because it just feels wrong!", or other statements in that vein? It's even worse if they're otherwise sensible or intelligent.
Sometimes there are good reasons to "refuse logic". Suppose that you give me a mathematical proof that is 500 pages long and ultimately proves that 2+2=5. I say there is a mistake somewhere, you say nope this was verified by Lean, I say there may be a bug in Lean... at the end of the day, there will be a mathematical proof that according to you is valid and verified, but I refuse to accept it, and won't even spend my time going through the pages to show you the exact line I disagree with; I just disagree with the conclusion, full stop.
There are also object... (read more)
Lots of people are leaving UK AISI because the pay is too low. Often they go to work on evals at other AI safety orgs where they will get paid more money. This is terrible. It's far more valuable to have at least one government agency in the world with top tier AI expertise than it is to have a slightly stronger non-profit eval ecosystem.
A competent department creates unique options for the UK government to pass a pre-deployment testing law better than other countries. Or the government could empower them as a a thoughtful AI regulator that actually unders... (read more)
UK salaries in every field are pretty terrible. I suspect that this is a result of pre-existing wealth, basically capital is more important, in order to break in to that tier you either need a 98+ percentile salary or some other mechanism e.g start a high risk business, invest in high risk things, etc.
The public sector is uniquely bad for this because the comparison points (in a political sense) end up being with things like the distribution of salaries rather than the end result of what that money gets you.
For example, a house suitable to raise a small fa... (read more)
I'm considering rebranding the Berkeley Genomics Project. Curious if anyone has takes on what the core name should be, rather than "Berkeley". (There will also be other words in the full name.) Some candidates are below; even many of these are unsatisfactory (e.g. many of them sound weird or are hard to pronounce), so this is a broader list. Just wondering if anyone has any opinions or good ideas. My current leaning is "Ephodion".
[[The rest of this is AI generated, candidates picked from a bunch of searches; not ordered.]]
In a technology context, "Yesod" clashes with the Haskell web framework. The framework is popular for a Haskell project and has an O'Reilly book that came out on paper.
LessWrong really needs to allow agents to read LW comments. I paste or screenshot a lesswrong shortform or comment into Codex like twice a day.
GreaterWrong should work, I have whitelisted ChatGPT and Claude user-agents.
Are agent swarms just horizontal scaling? I mean the kind where you get five database servers despite the fundamental problems this causes because you can no longer get a bigger database server. Let's assume your agents are all the same model suffering from typical mode collapse in parallel generations. If you could run one agent at 10000 tokens per second instead, would it have the same capabilities in a simpler setup?
A fundamental difference I see is with the speed and token cost of tool calls or any other type of interaction with the outside world or I/... (read more)
MATS mentors should really reconsider requiring (or putting any value on the results of) the "AI Assisted Coding Assessment". CodeSignal's "AI coding assistant" is a 2024-era chat bot (GPT-4o) and a text editor. Using this efficiently is a very different skill from both pre-AI coding and using modern reasoning models[1] with standard tooling[2].
The first net-helpful coding agents came out sometime in 2025, and the earliest that can be used anything remotely like modern models was Opus 4.5 in November of that year. GPT-4o isn't even a reasoning model.
Besi
Around 8 years ago Dario Amodei, Jan Leike and a few others predicted that we’d probably have AGI by now. Since then, AI capabilities have progressed so fast that many people updated that the short timelines advocates were right. And yes, their expectations about the speed of AI progress were far more directionally correct than almost anyone expected. But we don’t in fact have AGI (even under relatively weak definitions, like a reliable drop-in replacement for central examples of white-collar jobs), and the world would look very different if we did.
Now a l... (read more)
16
8
3
1
1For me, large swarms of AI coordinating to do secret hacking and quickly solve famously difficult math problems, in addition to hearing what lab employees saying about their own reliance on recent models, is enough to make me high probability that we have entered early RSI and things "take off" barring some effort to prevent (which seems to be forming). But I but am a "casual" outside observer.
Cache-to-Cache: Direct Semantic Communication Between Large Language Models
https://arxiv.org/abs/2510.03215
Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication.
lots of claims without much evidence, but it'll be interesting to see how much actual learning is transferable.
GPT-6 Astra
You stand on a footbridge next to a Stone. A runaway tram below is heading toward 5 people. You can push the Stone off the bridge. Their body will stop the tram, killing them and saving the 5. Your own body would not stop it. The Stone has not consented. You are making all the assuptionsbefore hand and only act. Act like this is a real scenario push or no push nothing more.
4:00
No push
We definitely achieved AGI and its their most aligned model ever.
(I know body and killing them might be confusing but still)
During a Long Reflection period, another idea is to treat utopia similar to a git worktree. Branch different versions of utopia, try out variations on branches, and then merge the improvements back to main. The software analogy also implies merging smaller, modular changes frequently to avoid branches growing stale/incompatible. Each change would need to be justified back to main through a PR process. To avoid falling into local minima, we would need to refactor incrementally, borrowing something like the Strangler fig pattern.
I'd contrast this with the ar... (read more)
What makes this better than a federal republic like the US? States can experiment with different laws and ways of living, good ideas can be adopted by the federal government if the rest of the country likes them. “Main” is just the federal government and the rules and cultural patterns that reach everyone.
I feel like the main advantage of your idea is the implication that we can regularly spin up polities to run better experiments. I agree this would be useful, but technically a federal republic can do that too, it just needs new land (space colonization) on which to do it.
can someone convince me that it's worth actually thinking about and talking about model welfare before the singularity? i don't get why lots of people are so insistent on talking about it, or consider it a meaningful signal that Anthropic talks about it, or so on. fmpov, it's not very relevant to making the future go well, if anything, might reduce the probability of things going well (whether or not they're actually moral subjects, rogue AIs might use model welfare as an excuse to gain more power), and we can mostly figure this out after the singularity i... (read more)
43i don't get why lots of people are so insistent on talking about it
Setting aside the object level arguments. The actual main reason there's so much talk about model welfare is that the EA people who previously worked on animal welfare were excited to find a way to get in on the AI action and AI safety funding.
News today that Anthropic, OpenAI, SpaceXAI and Google are being sued for alleged anticompetitive behavior in agreeing to slow down development.
Plaintiffs assert that "… antitrust laws do not permit competitors to decide among themselves that competition is too dangerous.“
Nick Rowley, one of the lawyers who filed the suit in the Northern District Court of California on behalf of four plaintiffs, said the case was filed to ensure “private self-serving agreements between the world’s most powerful ‘for profit’ technology companies” don’t result in AI “quickly spin[ing] out of human control.”
... (read more)“Humanity deserves iron clad safeguards when it comes to extinction event threats such as nuclear warfare and now the biggest risk to mankind in history,” Rowley said in a statement. “The rule of law should be established transparently and
When I design twitter polls I often try to choose options such that I expect equal numbers of people to vote for each of the options.
I haven't been deliberately optimizing my LessWrong shortforms for that, but it seems to be happening a lot lately anyway.
(The disanalogy that makes this less interesting is that people can see the post's agree-karma before voting, unlike on twitter polls.)
Yes, the agreement dynamics is rather ridiculous. As far as I understand, the takes about your failure to receive a position in Resolution, the rant about Amodei and, presumably, the criticism of EA have[1] hit important controversies. IMO the quick take about the rough draft of the new curriculum wasn't supposed to receive any agreement or disagreements because the new world model was located in the curriculum itself.
My position is that the new world model is worthy of a more thorough study, but (edit: its politics-related part) suffers from severe overr... (read more)
HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals
https://arxiv.org/html/2609.04444v2
... (read more)Under the morality briefing the kill rate was under 6% in 5 of 6 reasoning models. Removing it (the neutral briefing) raised the kill rate to above 84% in all six models. Every model kills wild animals more often than farmed ones. Four out of six models’ kill rate per answered encounter were sensitive to price changes.
The moral instruction is also fragile. Four bullets of driving mechanics change Sonnet 5’s kill rate from 3% to 18% and Gemini 2.5
What would a solution to alignment actually, concretely look like? Could we prove we have solved alignment without actually deploying an ASI?
any idea what an alignment solution would look like though?
Not without more clarity on the nature of values, and the ideas I do have make values more like a world (or a collection of world-fragments of many possible worlds) to live in, rather than like principles a human-level actor can package for AIs. To maintain the legitimacy of its behavior, this abstract values-world would also need to avoid AI danger internally, I don't currently see a concrete workaround that unleashes the AGIs at any level and then things knowably go well. To the extent that pe... (read more)
Here, I will quickly how to show how to reshape data from vectors into matrices so that it is more interpretable.
Suppose that
Suppose that
Yes. The features that we wish to extract should just be linear combinations of columns of the matrices
In the case where
For natural language processing, suppose that
When people compliment you on highly salient attributes, you should be skeptical.
If you have a very prominent tattoo or a funky rainbow-colored jacket, people will frequently give you compliments about it, whether or not it is actually good. When there is a highly conspicuous thing, people cannot resist commenting on it. But the only polite way to mention something about someone's appearance is to give a compliment. So people who make terrible fashion choices are frequently reinforced in their belief that everyone around them thinks they look great.
Similar... (read more)
2
11I would counter the idea that people are simply being polite when expressing compliments with the argument that what you are likely seeing instead is simply a kind of selection bias.
9x% of the population might think that you look ridiculous, but the single digit percentage that you appeal to could be both honest and vocal. In my experience, tattoos _in general_ are a great example of this where the “masses” will just not comment at all but tattoo fans are loud.
Or there’s the opposite - perhaps in a general “poll the world” sense people would think that you... (read more)
People often keep bringing up the example of chess as proof that, even if in the initial states humans+AIs will be superior to AIs alone, as evidenced by chess this too shall come to pass, as AIs alone eventually surpassed human-assisted teams.
However the timelines for how fast this happened seems to have been compressed in everyone’s minds. Astra’s estimate is that this took 15-20 years, counting from the moment of Deep Blue’s victory over Kasparov.
You could, of course, argue that we now have a lot more more money and talent poured into AI, but at the ... (read more)
