GooseVowel — LessWrong

Frontier models state different decision theory preferences depending on who's asking

I'm not so sure these act as controls, because they don't actually capture situations with no consensus.

Irrealism/realism: There is indeed something approaching consensus among academic experts and among LW readers (otherwise, the header "academia vs. LW divide" wouldn't make sense). Most people who know about the issue can predict your take based on if you are a LW reader or an academic. I claim P(doom) has this property as well (exhibit A: mainstream academics and policy ignoring rationalist warnings about AI until approximately last week).

IMO the bett... (read more)

Jan_Kulveit's Shortform

GooseVowel2mo1-1

I agree with basically everything in this comment, except maybe this part: "The emotional impact makes it less easy to treat your belief in it as a tool for impressing your friends and irritating your enemies, rather than something to actually worry about and try to solve if it's real."

I just don't think the argument goes through: "The emotional impact being stronger, makes it LESS easy for you to treat your belief as a tool for impressing friends/irritating enemies". (Citing HPMOR: "For it is a sad rule that whenever you are most in need of your art as a... (read more)

Pretraining data, not verifiability, is why LLMs are especially good at math (and coding)

GooseVowel2mo10

I also have question2.

IIUC, The method Ryan describes is "generate a bunch of programs" and then "filter those". He wrote:

> The distribution of programs you are searching over [after generating a bunch of them] has to be pretty close to the right program for Best-of-6k to work at all: if you did best-of-6k for random python programs, this would not work!

I agree that it's way better than random... but is it better than the human literature? E.g., If you asked me to filter 6,000 NLP papers, I'd expect a great many of them to be "basically true" and a g... (read more)

Just the facts, ma'am!

GooseVowel2mo10

Thanks for this, as I was reading I thought of this: "I cannot remember the books I have read, anymore than the meals I have eaten; Even so, they have made me". I used to be very disturbed by not being able to remember all the facts and details I've read, and I found this quote quite soothing (I haven't fact-checked, but it was attributed to Ralph Waldo Emerson).
I interpret questions like "Do you think X will happen?" as meaning something like "Please report your posterior on event X, after all the updates you've had so far". People can answer with just t... (read more)

The Talker Does Not Control The Doer (in Current AIs)

GooseVowel2mo31

I have two clarification questions.

Question: IIUC the "talk-y part" and "the do-y" part are sub-parts of a single model instance (circuits, or idk some sub-model package of computation). That is, you're NOT suggesting that the claude app is splitting up work into subagents under the hood, so that 3 models in a trench coat appear to be just 1 model. Do I have that right?

Question: How could this have turned out any different? What else could you reasonably expect to find? Is the alternative "a thinking thing that is aware of all its computations, at all gr... (read more)