This is a special post for quick takes by Raemon. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
This is an experiment in short-form content on LW2.0. I'll be using the comment section of this post as a repository of short, sometimes-half-baked posts that either:
don't feel ready to be written up as a full post
I think the process of writing them up might make them worse (i.e. longer than they need to be)
I ask people not to create top-level comments here, but feel free to reply to comments like you would a FB post.
most of the value is not coming from the 200-line instruction file, it's coming from the human judgment doing selection/pruning/etc.
It's still early days at this point. I don't think anyone can be confident regarding the potential quality of selection/pruning/etc. possible given the current technology.
I think if I was going solely based on your comments, I would not believe that creation of a paper that could go in a 1st quartile social science journal is possible with current technology. Yet experts appear to believe it is, in fact, possible. I would encourage you to practice the virtue of lightness on this topic:
The third virtue is lightness. Let the winds of evidence blow you about as though you are a leaf, with no direction of your own. Beware lest you fight a rearguard retreat against the evidence, grudgingly conceding each foot of ground only when forced, feeling cheated. Surrender to the truth as quickly as you can. Do this the instant you realize what you are resisting, the instant you can see from which quarter the winds of evidence are blowing against you. Be faithless to your cause and betray it to a stronger enemy. If you regard evidence as a constraint and seek to free yourself, you sell yourself into the chains of your whims. For you cannot make a true map of a city by sitting in your bedroom with your eyes shut and drawing lines upon paper according to impulse. You must walk through the city and draw lines on paper that correspond to what you see. If, seeing the city unclearly, you think that you can shift a line just a little to the right, just a little to the left, according to your caprice, this is just the same mistake.
Regarding investment of effort:
And current LLMs are not actually good enough to act as a reliable filter in a way that can be scalably automated, at least not with amounts of effort that make sense to invest given the steady march of progress.
I think it could make a lot of sense for someone to invest that effort, if they want to create differential technological development in favor of alignment research. I expect your current course of action will effectively take that option off of the table in practice.
If you wait until it's manifestly obvious that AI agents can help with research, you're essentially creating differential technological development against AI alignment research, relative to other field that continue to publish solely on the basis of quality.
If you must discourage AI, at least you could use some sort of very sensitive leading indicator such as a weekly competition for best AI-generated alignment post, so AI posts are encouraged the instant the technology is good enough to help us with alignment. (Specifically, each week the winner of the previous week's competition could be submitted masquerading as an ordinary LW post to see how it scores, so that way you're collecting 1 data point per week about the state of the technology, while allowing a max of 1 AI "spam" post per week.)
But instead I would suggest you simply discourage posts if you can tell it's AI, rather than using the honor system.
This is an experiment in short-form content on LW2.0. I'll be using the comment section of this post as a repository of short, sometimes-half-baked posts that either:
I ask people not to create top-level comments here, but feel free to reply to comments like you would a FB post.