Harebrained alignment idea: LLMs can't be trusted to assist with alignment research because it's too easy to get them to say what you want to hear (e.g. make you think you've solved a safety problem when you haven't). Therefore, AI companies should train a distinct LLM that doesn't go through RLHF or any other process by which it's reinforced based on how much people like its responses.
This doesn't fix the problem that next-token-generators are architecturally better at generating plausible-sounding statements than true statements, but it does help with sycophancy etc.
Another related idea is to tune an LLM really hard toward criticism: make it work as hard as it can to come up with reasons why you're wrong.
For what it's worth, Claude will (unprompted) mention why a technique might be interesting but doesn't solve the hard problems of alignment, but this could depend on the conversation. This only work if it knows that a technique won't work, and it's frequently too optimistic about new techniques though.