I operate by Crocker's rules.
Thanks, that updates me. I've been enjoying your well-informed comments on big training runs, thank you!
On priors I think that Google Deepmind is currently running the biggest training run.
The Variety-Uninterested Can Buy Schelling-Products
Having many different products in the same category, such as many different kinds of clothes or cars or houses, is probably very expensive.
Some of us might not care enough about variety of products in a certain category to pay the extra cost of variety, and may even resent the variety-interested for imposing that cost.
But the variety-uninterested can try to recover some of the gains from eschewing variety by all buying the same product in some category. Often, this will mean buying the cheapest acceptable product from some category, or the product with the least amount of ornamentation or special features.
E.g. one can buy only black t-shirts and featuresless cheap black socks, and simple metal cutlery. I will, next time I'll buy a laptop or a smartphone, think about what the Schelling-laptop is. I suspect it's not a ThinkPad.
"Then let them all have the same kind of cake."
And: yes, the games weren't normalized to be zero-sum.
I wrote a short reply to Dagon, maybe that helps.
Otherwise I might write up a full post explaning this with examples &c.
Updated the link to the actual code. I computed the equilibria for the full game, and then computed the payoff per equilibrium for each player, and then took the mean for each player. I did the same but with the game with one option removed. The number in the chart is the proportion of games where removing one option from player A improved the payoff (averaged over equilibria).
If the number is >0.5, then that means that for that player, removing one option from A on average improves their payoffs. (The number of options is pre-removal). I also found this interesting, but the charts are maybe a bit misleading because often removing one option from A doesn't change the equilibria. I'll maybe generate some charts for this.
I'll perhaps also write a clearer explanation of what is happening and repost as a top-level post.
How Often Does Taking Away Options Help?
In some game-theoretic setups, taking options away from a player improves their situation. I ran a Monte-Carlo simulation to figure out how often that is the case, generating random normal form games with payoffs in , removing a random option from the first player, and comparing the Nash equilibria found via vertex enumeration of the best response polytope (using nashpy)—the Lemke-Howson algorithm was giving me duplicate results.
Code here, largely written by Claude 3.5 Sonnet.
I find the Thompson hack very fascinating from an agent foundations perspective. It's basically a small version of reflective stability in the context of operating systems.
I used to find compilers written in their own language kind of—…distasteful, in some way? Some of that is still present, because in reality it's just that the bootstrapping chains become very long and difficult to follow. But I think a small part of that distaste was the worry that Thompson hack-style errors occur accidentally at some point, and are just propagated through the bootstrapping chain. After thinking about this for a few seconds this was of course patently ridiculous.
But under this lens reflective stability becomes really difficult, because every replicating/successor-generating subsystem needs to be adapted to have the property of reflective stability.
E.g. corrigibility is really hard if one imagines it as a type of Thompson hack, especially under relative robustness to scale. You don't just get a basin of Thompson-hackness when writing compilers and making mistakes.
I think the splash images are >95th percentile of AI generated images in posts in beauty, especially as they still carry some of the Midjourney v1-v3 vibe, which was much more gritty and earnest (if not realistic) than the current outputs.
I really like some of the images people have used for sequences, e.g. here, here, here and here. Wikimedia has tons of creative commons images as well which I'd use if I were more into that.
The current state of the art for salary negotiations is really bad. It rewards disagreeableness, stubornness and social skills, and is just so unelegant.
Here's a better way of doing salary negotiation:
Procedure via a two-sided sealed-bid auction, splitting the difference in bids[1]:
rmin and rmax do not need to be positive! It might be that the potential employee likes the project so much that they set rmin to zero or even negative—an exceptionally great idea might be worth paying for. Or rmax might be negative, in that case one party would be selling something.
I'm not aware of anyone proposing this kind of auction for salary negotiation in particular, Claude 3.5 Sonnet states that it's similar to Vickrey auctions, but in this case there is no second price, and both parties are symmetrical.
I think that the setup described is probably not incentive-compatible due to the Myerson-Satterthwaite theorem, like the first-price sealed-bid auction. (I still think it's a vast improvement over the current state of the art, however). For an incentive-compatible truthful mechanism the Vickrey-Clark-Groves mechanism can be used, but I'm still a bit unsure how the subsidising would work. ↩︎