https://soniafpearson.substack.com/p/capitalizing-untethered-ai-agents Tyler Cowen and Sonia Farrell Pearson argue that in an economy of "untethered" AI agents it might be necessary to enforce agents as having a unique identity, and make them hold capital that can be threatened by law, in order to make it possible to penalize the agent itself for torts when there's no human individual or corporation that can be held responsible.
this makes some sense, but i wonder whether we should simply assume that "untethered" agents will be a thing, as opposed to other possibilities like "agents will always have to be associated with an individual or firm" or "there will never be an agent economy".
it seems more like, if you want to have a maximally autonomous agent economy, you will need to have legal structures that can hold agents accountable, and this is one initial plausible stab at how that could look. this is a case you make if you are bullish about a world of agents.
That's really the only happy endgame for AI safety regulation; we must define what we consider safe vs. unsafe, and require that AI-ish technologies are safe by that standard; and hopefully, that standard actually is indicative of safety from catastrophic risks in real life. (Even on the "if anyone builds it everyone dies" view, we have to have an operationalizable, enforceable definition of what "it" is!)
then, you can do basic research, inspired by the "use case" of getting to a safety case. you try to understand how AIs even work and what you can prove about them, with the ultimate but indirect aim of being able to prove that they don't do a "bad thing" (or do the bad thing with sufficiently low probability.)
the example given for "use case inspired basic research" is the physics of radio transmitters. but i think the more obvious example is human biology. almost everybody studying the human body is keeping in mind an ultimate goal of developing medical technology.
it's important to note that people from different disciplines (here, basic research, applied research, and policy) are going to think differently, and look at the same objects with different priorities, and that's okay.
https://nischmainali.github.io/blog/nonlinear-direction-learning/ a nice toy model of how feature learning in a two-layer neural network involves a phase transition in the weights as the training data grows. we consider a Bayesian model of the neural net that has exactly the same structure, except that the weights after a training run are one sample from the posterior distribution of the weights, and training updates the posterior.
this is an example of a statistical mechanics toolkit for analyzing deep learning; the posterior is the Gibbs-Boltzmann distribution of a system whose energy is the squared training loss plus a prior. temperature in the stat-mech formulation corresponds to noise in SGD.