at first she had qualms literally called "Effective Evil"

I think this sentence is missing "working for an organization".

I found this post interesting but I think there is something wrong with it, even though I estimate that its central point has value. My remarks focus on "this post as an advice" rather than the "phenomenon explanation" part of the post.

Perhaps the first thing I should say is that I agree some people have a tendency to think too much in certain situations. To delay the first try for too long and to waste a lot of time optimizing entire lines of though that will be revealed to be worthless after five minutes of concrete work. I am one of those people and I have spent time thinking (eh) about when and how this must be improved and corrected.

But I do not think a general cheer for the policy of "acting before thinking" is a good thing. I can cite two examples from my personal life, in the last 10 days, where a dozen hours or more were wasted because someone did not spend half an hour thinking when it was "obviously" sensible to do so. The first one involves writing code without thinking, creating bad code that had to be managed afterward (technical debt), the second involves personal relationships and is private (sorry).

I estimate that this post presents danger on the same scale as the gain it might deliver. In fact I expect this post to be a net negative in terms of direct advice. Ideally, the wisdom found in this post would have its place in a more complete framework on decision processes. As it stands, I am tempted to compare this advice to "to walk, move your right leg forward", dangerously misleading and incomplete without the complementary advice "also move your left leg".

Perhaps a better immediate advice would be.

When applicable and when your actions will not have non-trivial lasting consequences think just a little then act, make an attempt. Only after your have tried enough can you think again, if you believe it to be useful.

If we apply this advice to different timescales we get both your second and third example. We however do not get the first, insofar as there can be lasting consequences to a botched application.

A few typos.

• How’s intuitively going to be better --> Who's intuitively going to be better
• igure out” that they need to to help and how. --> igure out” that they need to help and how.
• very single job as though I’d be a life-and-death decision whether or not to apply --> very single job as though it'd be a life-and-death decision whether or not to apply

Thank you. Having one answer definitely feels better than none. I think I will delay the project a bit and gather views until I have amassed enough. Hopefully recommendations can pile up with time, i think I receive some periodically. I will write down your recommendation and they will be part of my effort, though probably not right now.

Recently, Yudkowsky has been recently talking about third countries stealing

Should probably be

Recently, Yudkowsky has been talking about third countries stealing

I know this comment is a few months old, but an answer might still be helpful.

I think the following quote from the article answers your question.

From Yudkowsky’s perspective, I gather, talking about human rationality without saying anything interesting about AI is about as difficult as talking about AI without saying anything interesting about rationality.