Important reminder, and an enjoyable read!
re: "Bugs outside the kernel", isn't the idea of having a trusted kernel that the kernel will reject any false proofs caused by bugs outside of it? My understanding was that the elaborator can introduce a flaw in a top-level statement being proven, if it doesn't do a faithful translation of the statement or the definitions involved, but any bugs in the proof or the lemmas it depends on shouldn't make it past the kernel without a separate kernel bug.
I also wouldn't expect the compiler and build system to affect the... (read more)
Second group of headlines on reuters.com, after Iran / Houthis: OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup
I am curious when scores in some areas go backwards, despite some clear large gains (and did they try that experimental scaffold again?)
The part where they remove the experimental scaffold is the only thing in this table that goes backwards from Opus 4.* to Mythos, right? (Lower is better for MSE.)
Some things did get worse from 4.5 to 4.6, which I agree is interesting.
What I mean is that for branches to affect each other, the branches must become completely not separated: every difference between the branches must be erased, including the ion and also the atoms in all the observers. This makes communication meaningless, because there's no longer anything the sender knows that the receiver doesn't also know.
Inter-branch communication while preserving separate branches shouldn't be possible under the standard many-worlds interpretation, because it violates linearity.
"Reality is amplitudes flowing between configurations" is helpful for understanding conceptually why MWI (and quantum mechanics generally) predicts this won't work. For branch 2 to influence branch 1, the two branches' amplitudes would have to flow to the same configurations. So suppose Alice#1 (with high probability) receives a message, and is still aware that she is #1 and didn't compute the mess... (read more)
This strikes me as a core problem of AI alignment, rather than something lost amid the struggle to solve alignment.
We don't yet know how to build human-aligned AI. But I expect one property any human-aligned AI must have is that it actually listens to humans in some way. Some of the most promising partial proposals for safe superintelligence involve quite a bit of creativity in translating human input into AI behavior (I'm thinking in particular about iterated distillation and amplification here).
For any good proposal to scale up AI to a benevolent sovere... (read more)