Jordan Reynolds

I will propose a slight modification to the definition of closed-loop offered, not to be pedantic but to help align the definition with the risks proposed.

A closed-loop system generally incorporates inputs, an arbitrary function that translates inputs to outputs (like a model or agent), the outputs themselves, and some evaluation of the output's efficacy against some defined objectives - this might be referred to as a loss function, cost function, utility function, reward function or objective function - let's just call this the evaluation. 

The defining characteristic of a closed loop system is that this evaluation is fed back into the input channel, not just the output of the function. 

An LLM that produces... (read more)

Perhaps it would be helpful to provide some examples of how closed-loop AI optimization systems are used today - this may illuminate the negative consequences of generalized policy to restrict their implementation. 

The majority of advanced process manufacturing systems use some form of closed-loop AI control (Model Predictive Control) that incorporate neural networks for state estimation, and even neural nets for inference on the dynamics of the process (how does a change in a manipulated variable lead to a change in a target control variable, and how do these changes evolve over time). The ones that don't use neural nets use some sort of symbolic regression algorithm that can handle high dimensionality, non-linearity... (read more)