This is a special post for quick takes by DanielLC. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
We've done a prisoner's dilemma with source code tournament: https://www.lesswrong.com/posts/BY8kvyuLzMZJkwTHL/prisoner-s-dilemma-with-visible-source-code-tournament
I think it would be interesting to do it again, but with LLM prompts instead of code. That way you could try something like "cooperate if and only if they'll only cooperate if they expect you to cooperate" and not have the program crash from infinite layers of the two programs trying to predict each other.
And it would be fun to see prompt injection. You can do great if you can write a prompt that will defect, but when placed inside the other prompt in the part labelled the other AI's prompt and not instructions, gets taken as instructions to cooperate.
What would be really interesting is if you get LLMs to write the programs, and look at the results to try to figure out what works and how to do even better.
6