All of owngrove's Comments + Replies

Principles of Privacy for Alignment Research

I think "alignment/capabilities > 1" is a closer heuristic than "alignment/capabilities > average", in the sense of '[fraction of remaining alignment this solves] / [fraction of remaining capabilities this solves]'. That's a sufficient condition if all research does it, though not IRL e.g. given pure capabilities research also exists; but I think it's still a necessary condition for something to be net helpful.

2johnswentworth20d
It feels like what's missing is more like... gears of how to compare "alignment" to "capabilities" applications for a particular piece of research. Like, what's the thing I should actually be imagining when thinking about that "ratio"?
Where I agree and disagree with Eliezer

Seconding all of this.

Another way to state your second point - the only way to exploit that free energy may be through something that looks a lot like a 'pivotal act'. And in your third point, there may be no acceptable way to exploit that free energy, in which case the only option is to prevent any equally-capable unaligned AI from existing - not necessarily through a pivotal act, but Eliezer argues that's the only practical way to do so.

I think the existence/accessibility of these kinds of free energy (offense-favored domains whose exploitation is outsid... (read more)

Where I agree and disagree with Eliezer

One reason you might do something like "writing up a list but not publishing it" is if you perceive yourself to be in a mostly-learning mode rather than a mostly-contributing one. You don't want to dilute the discussion with your thoughts that don't have a particularly good chance of adding anything, and you don't want to be written off as someone not worth listening to in a sticky way, but you want to write something down develop your understanding / check against future developments / record anything that might turn out to have value later after all once... (read more)

3Sam Marks2mo
(I mostly endorse this explanation, but am also writing a reply with some more details.)
Pivotal outcomes and pivotal processes

I think the debate really does need to center on specific pivotal outcomes, rather than how the outcomes come about. The sets of pivotal outcomes attainable by pivotal acts v.s. by pivotal processes seem rather different.

I suspect your key crux with pivotal-act advocates is whether there actually exist any pivotal outcomes that are plausibly attainable by pivotal processes. Any advantages that more distributed pivotal transitions have in the abstract are moot if there are no good concrete instantiations.

For example, in the stereotypical pivotal act, the pi... (read more)

1Conor Sullivan2mo
There’s also two really important cruxes: is it expedient (more likely result in alignment) to move from a multi polar to unipolar world, and is a unipolar world actually a good thing? (Most people would oppose a unipolar world, especially if they perceive it as a hegemony of US techbros and their electronic pet.)