Feels like something has gone wrong way before when one cares more about money than survival of the human race.
If a man's judgement is really swayable by equity one cant stop to wonder whether he is the right man for the job in the first place.
3Did you see the movie before ?
I mean paperclip maximization is of course much more memetic than 'tiny molecular squiggles'.
So indeed if you define coherence as the negative of arbitrage value.
There is a pretty close relation between thermodynamic free energy and arbitrgable value and degree to which an entity can be money pumped.
You might also be interested in :
https://www.lesswrong.com/posts/3xF66BNSC5caZuKyC/why-subagents
Looking over the comments, some of the most upvoted comments express the sentiment ththat Yudkowsky is not the best communicator. This is what the people say.
1For my interest, for these reallife latents with many different pieces contributing a small amount of information do you reckon Eisenstat's Condensation / some unpublished work you mentioned at ODYSSEY would be the right framework here?
100 percent this. There is this perpetual miscommunication about the word "AGI".
"When I say AGI, I really mean a general intellignintellignence not just a new app or tool."
Reward is not the optimization target.
The optimization target is the Helmholtz free energy functional in the conductance-corrected Wasserstein metric for the step-size effective loss potential in the critical batch size regime for the weight-initialization distribution as prior up to solenoidal flux corrections
10
7
5
3
2
2
1
Happy to hear ILIAD felt productive!
When we started ILIAD this was from a felt sense that there is in fact much more commonality and convergence in research directions than commonly assumed.