As with the last article, I think this is almost entirely incoherent/wrong; as with the last article, I'm strong-upvoting it anyway because I think it makes ~1.5 good and important points I've not seen made anywhere else, and they're worth the chaff. (I'd go into more detail but I don't want anyone leaning on my summary instead of reading it themselves.)

. . . is there a reason this is a link to a google doc, instead of a copypaste?

^[1]

1. ^{^}

   and add footnotes.

"Using information during Training and/or Evaluation of models which wouldn't be available in Deployment."

. . . I'll edit that into the start of the post.

Thanks for a good one

I'm glad you feel that way about this scenario. I wish I did . . .

(For future reference, on the off-chance you haven't seen it: there's a compilation of all the past scenarios here, handily rated by quality and steamrollability.)

One thing that perhaps would make it easier was if the web interactive could tell whether or not your selection was the optimal one directly, and possibly how higher your expected price was than the optimal price (I first plugged mine in, then had to double check with your table out here)

. . . huh. I feel conflicted about this on aesthetic grounds - like, Reality doesn't come with big flashing signs saying "EV-maxxing solution reached!" when you reach an EV-maxxing solution - but it does sound both convenient to have and easy to set up. Might try adding this functionality to the interactive for the next one; would be curious to hear what anyone else who happens to be reading this comment thinks.

Anyway, greetings, and looking forward to seeing the next one.

Good to have you on board!

I really wish I could simultaneously strong-upvote and strong-downvote the "agree" thing for this reply. I think most of the description is horoscope-y flattery, but it doesn't have zero correlation with reality: I do think lsusr's writing is ninety-something-th percentile for 

a kind of minimalist clarity that leaves room for the reader to reflect and draw their own conclusions

and at least eighty-something-th percentile for

willing to question common assumptions within the rationalist sphere

while afaict there's nothing in the description that's the opposite of the truth.

(I also think there's something interesting about how the most meaningful/distinguishing lines in the description are the ones which could be most easily rephrased as criticisms. Does "describe X as neutrally as possible" or "describe the upsides and downsides of X" produce better LLM results than "describe X"?)

I used to implicitly believe that when I have a new idea for a creative(/creative-adjacent) project, all else being equal, I should add it to the end of my to-do list (FIFO). I now explicitly believe the opposite: that the fresher an idea is, the sooner I should get started making it a reality (LIFO). This way:

• I get to use the burst of inspired-by-a-new-idea energy on the project in question.
• I spend more time working on projects conceived by a me with whom I have a lot in common.

The downsides are:

• Some old ideas will end up near the bottom of the pile until I die or the Singularity happens. (But concepts are cheaper than execution, and time is finite.)
• I get less time to polish ideas in my head before committing pen to paper. (But maybe that's good?)

Thoughts on this would be appreciated.

Something low complexity, repeatable, non-addictive, non-time consuming.

I think a lot of people have a habit like that, and it's different things for different people.

I think it's meditation

Beware.

(Like, from what I hear you're not wrong, but . . . y'know, Beware.)

If anyone reading this wants me to build them a custom D&D.Sci scenario (or something similar) to use as a test task, they should DM me. If the relevant org is doing something I judge to be Interesting and Important and Not Actively Harmful - and if they're okay with me releasing it to the public internet after they're done using it to eval candidates - I'll make it for free.

I think you can address >95% of this problem >95% of the time with the strategy "spoiler-tag and content-warn appropriately, then just say whatever".

Is there value in seeking out and confronting these limits,

Yes.

or should we exercise caution in our pursuit of knowledge?

Yes.

. . . to be less flippant: I think there's an awkward kind of balance to be struck around the facts that

A) Most ideas which feel like they 'should' be dangerous aren't^[1].

B) "This information is dangerous" is a tell for would-be tyrants (and/or people just making kinda bad decisions out of intellectual laziness and fear of awkwardness).

but C) Basilisks aren't not real, and people who grok A) and B) then have to work around the temptation to round it off to "knowledge isn't dangerous, ever, under any circumstance" or at least "we should all pretend super hard that knowledge can't be dangerous".

D) Some information - "here's a step-by-step-guide to engineering the next pandemic!" - is legitimately bad to have spread around even if it doesn't harm the individual who knows it. (LWers distinguish between harmful-to-holder vs harmful-to-society with "infohazard" vs "exfohazard".)

and E) It's super difficult to predict what ideas will end up being a random person's kryptonite. (Learning about factory farming as a child was not good for my mental health.)

I shouldn't trusted with language right now.

I might be reading too much into this, but it sounds like you're going through some stuff right now. The sensible/responsible/socially-scripted thing to say is "you should get some professional counseling about this". The thing I actually want to say is "you should post about whatever's bothering you on the appropriate 4chan board, being on that site is implicit consent for exposure to potential basilisks, I guarantee they've seen worse and weirder". On reflection I tentatively endorse both of these suggestions, though I recognize they both have drawbacks.

1. ^{^}

   For what it's worth, I'd bet large sums at long odds that whatever you're currently thinking about falls into this category.