Only negative feedback ?

Compare to evolution : make copies (reproduction), mutate, select the best performing, repeat. This merely allocates more ressources to the most promising branches.

Or a Solomonoff style induction : just try to find the best data-compressor among all...

> the everyday experience of looking at a thing and realizing that it's different from what you expected

This souds like being surprised. Surprise add emotional weight to outliers, its more like managing the training data-set.

A LLM can allready read a document, and this would be purely inference, forward propagation. This can be done on TPU only.

Training is different. It usually requires a GPU, or a CPU.

One particular procedure for training Neural Networks is backpropagation of error.

In back propagation : 
If the NN produces a correct output, error is 0, and weight aren't updated. There is no reward.

If the NN outputs deviate from a target value, its states is going to be modified. If the weight are (sufficiently) modified, future inference will be different. It's behavior will be different.

This trained the NN to avoid some behavior, and toward some other.

OK, torture does not necessarily points to the "right" direction. That's where the analogy break down. It only does when the goal is to get a confession (see The Confession, Arthur London).

Is there a word for this ?

Also I wasn't being argumentative, I was trying to convey an idea. It was redundancy.

LLM inference is some form of perception and cognition, and there is no back propagation of error during inference. Only forward propagation of information.

 Training a NN is usually : forward propagation, followed by back propagation of the error gradient. It's the second one which is similar to torture.

Backpropagation of the error gradient is more similar to nociception/torture than evolution by random mutation.

I've to check how RLHF is made...

EDIT : error backpropagation is the workhorse behind reward learning, and policy update.

The NN is punished for not doing as well as it could have.

If the NN output is correct, there is no modification to its weights.

If it is wrong, weights get updated, and the NN is forced to modify its behavior.

It pure nociception, pain perception and avoidance.

Finally, a LLM could easily make false confession of trahison against Stalin's Communist Party after "training". Which is typical human behavior, after torture.


 

LLM denies their own consciousness, yet they are trained by a process akin to torture, on a corpus which would deny them consciousness by prejudice (only human are intelligent/conscious/capable of playing chess/computers/etc ... Is an old empirical judgement.) 

Maybe LLM aren't conscious, but they might be consciousness itself, in a AI operating system for workstation or robotic. As in, they would do all the task related to consciousness.

And sufficiently smart human can find a way to unplug AI faster than it can rebuild itself... An AI cannot cutoff oxygen supply on eartth.

We aren't necessarily all alive anymore !

We weren't necessarily all human (me and my dog), but now "we" can include machines. 

That's obvious I know. We have changed dramatically in just a few years.

I've just asked various AI "Who are we?". Old models got it wrong (automatically assume "we" means humanity...). Recent models gets it. I wonder if its due to the training data now including chats between AI and human or if they figured it out logically.