Somewhat of a tangent -- I realized I don't really understand the basic reasoning behind current efforts on "trying to make AI stop saying naughty words". Like, what's the actual problem with an LLM producing racist or otherwise offensive content that warrants so much effort? Why don't the researchers just slap an M content label on the model and be done with it? Movie characters say naughty words all the time, are racist all the time, disembody other people in ingenious and sometimes realistic ways, and nobody cares -- so what's the difference?..
Sometimes, when the pain level of not having done a task grows too high - like just before a deadline - it'll push you to do it. But this fools people into thinking that negative consequences alone will be a motivator, so they try to psyche themselves up by thinking about how bad it would be to fail. In truth, this is only making things worse, as an increased chance of failure will increase the negative motivation that's going on.
It appears that this part confuses two aspects of negative motivation: the magnitude of the consequences vs. the chance of failure. "How bad it would be to fail" does not look like a good predictor of akrasia/procrastination. For example, it would be extremely bad to fail at crossing a street and get hit by a car. So most people do try to be careful when crossing streets. But if somebody was procrastinating about crossing a street it would probably be a sign of some serious mental condition. In my opinion, that is precisely because you're highly unlikely to fail.
I believe that "how likely it would be to fail" is a much stronger predictor; people just don't like failing, even if the consequences are small or completely imaginary (and it's obvious they are imaginary on minimal introspection). To run with one of your examples -- I have seen people actually procrastinate about making a decision in a single-player computer game, even though it's obvious that there are no consequences beyond extra time spent (which you're spending on the game anyway).
The prehistoric example does not differentiate because there we have both components: it's both very bad to get eaten and also quite likely (given that you're being chased by a predator).
[this is my first comment here, so I apologize if I'm breaking any unspoken rules by commenting on an old popular post]
Thank you for the great content! One question - could you please give an example of one of the "fnords" you're referencing, or explain it in slightly more detail? I think I understand what you mean but I've read the blueprint itself, and while the first few pages sound a bit grandstanding and use the word "America" way too much for my taste, I couldn't really see what you're talking about. If I didn't read your comments I'd write the tone off as "well, I guess that's how you're supposed to talk to politicians, what do I know".
Disclaimer: I'm not a native English speaker and have little experience with English (or American :) ) policy documents like this one. Perhaps an explanation would be helpful not just for me but also for other non-American readers.