This is a special post for quick takes by Legionnaire. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Months ago I suggested that you could manipulate the popular LLMs by mass publishing ideological text online. Well this has now been done by Russia.
We should expect LLMs to get just as contaminated as Google search soon. Russia does it for ideological purposes, but I imagine that hundreds of companies already do it for commercial reasons. Why pay for advertisement, if you can generate thousands of pages promoting your products that will be used to train the next generation of LLMs?
Who is aligning lesswrong? As lesswrong becomes more popularized due to AI growth, I'm concerned the quality of lesswrong discussion and posts has decreased since creating and posting have no filter. Obviously no filter has been a benefit while lesswrong was a hidden gem, only visible to those who can see its value. But as it becomes more popular, i think it should be obvious this site would drop in value if it trended towards reddit. Ideally existing users prevent that, but obviously that will tend to drift if new users can just show up. Are there methods in place for this issue?
Specific example: lots of posts seem like rehashes of things that have already been plainly discussed, and the quick takes section, and discussion on Discord, do a great job of cutting down on this particular issue. So maintaining high quality posts is not a pipe dream!
Thanks for reminder! I looked at the rejected posts, and... ouch, it hurts.
LLM generated content, crackpottery, low-content posts (could be one sentence, is several pages instead).
Speculation: LLM Self Play into General Agent?
Suppose you got a copy of GPT4 post fine tuning + hardware to train it. How would the following play out?
1. Give it the rules and state of a competitive game, such as automatically generated tic-tac-toe variants.
2. Prompt it to use chain of thought to consider the best next move and select it.
3. Provide it with the valid set of output choices (like a json format determining action and position, similar to AutoGPT)
4. Run two of these against each other continuously, training on the results of the victor which can be objectively measured by the game's rules.
5. Benchmark it against a tiny subset of those variants that you want to manually program a bot with known ELO / have a human evaluate it.
6. Increase the complexity of the game when it reaches some general ability (eg tic tac toe variants > chess variants > Civilization 5 The Videogame variants)
Note this is similar to what Gato did. https://deepmind.google/discover/blog/a-generalist-agent/
This would have an interesting side effect of making its output more legible in some ways than a normal NN agent, though I suppose there's no guarantee the chain of thought would stay legible English unless additional mechanisms were put in place, but this is just a high level idea.