RL algorithms don't minimize costs, but maximize expected reward, which can well be unbounded, so it's wrong to say that the ML field only minimizes cost.
LLMs minimize expected log probability of correct token, which is indeed bounded at zero from below, but achieving zero in that case means perfectly predicting every single token on the internet.
The boundedness of the thing you're minimizing is totally irrelevant, since maximizing is exactly the same as maximizing $g (f (x))$ where g is a monotonic function. You can trivially turn a bounded function into an unbounded one without changing anything to the solution sets.
Even if utility is bounded between 0 and 1, an agent maximizing the expected utility will still never stop, because you can always decrease the probability you were wrong. Quadruple-check every single step and turn the universe into computronium to make sure you didn't make any errors.

This is very dumb, Lecun should know better, and I'm sure he *would* know better if he spent 5 minutes thinking about any of this.

[-]tailcalled1y20

RL algorithms don't minimize costs, but maximize expected reward, which can well be unbounded, so it's wrong to say that the ML field only minimizes cost.

Yann LeCun's proposals are based on cost-minimization.

[-]Razied1y20

Do you expect Lecun to have been assuming that the entire field of RL stops existing in order to focus on his specific vision?

[-]tailcalled1y40

I'm not sure he has coherent expectations, but I'd expect his vibe is some combination of "RL doesn't currently work" and "fields generally implement safety standards".

[-]Adam Jermyn1y62

Another objection is that you can minimize the wrong cost function. Making "cost" go to zero could mean making "the thing we actually care about" go to (negative huge number).

[-]tailcalled1y2-2

I don't think this objection lands unless one first sees why the safety guarantees we usually associate with cost minimization don't apply to AGI. Like what sort of mindset would hear Yann LeCun's objection, go "ah, so we're safe", and then hear your objection, and go "oh I see, so Yann LeCun was wrong"?

[-]Viliam1y1-8

One way to minimize costs is to kill all humans, then money loses all meaning, and the cost of anything is zero.

[-]FlorianH1y00

Dear Yan LeCun, dear all,

Time to reveal myself: I'm actually just a machine designed to minimize cost. It's a sort of weighted cost of deviation from a few competing aims I harbor.

And, dear Yan LeCun, while I wish it was true, it's absolutely laughable to claim I'd be unable do implement things none of you like, if you gave me enough power (i.e. intelligence).

∎.

I mean to propose this as a trivial proof by contradiction against his proposition. Or am I overlooking sth?? I guess 1. I can definitely be implemented by what we might call cost minimizationf^[1], and sadly, however benign my today's aims in theory, 2. I really don't think anyone can fully trust me or the average human if any of us got infinitely powerful.^[2] So, suffices to think about us humans to see the supposed "Engineers"' (euhh) logic falter, no?

^{^}
Whether with or without a strange loop making me (or if you want making it appear to myself that I would be) sentient doesn't even matter for the question.
^{^}
Say, I'd hope I'd do great stuff, be a huge savior, but who really knows, and, either way, still rather plausible that I'd do things a large share of people might find rather dystopian.

Moderation Log

LESSWRONG
LW

LESSWRONG
LW

19

Yann LeCun: We only design machines that minimize costs [therefore they are safe]

19

19