Thanks. Another reason to avoid anthropomorphic terms like "agent" is to avoid confusing lawyers, legislators and judges, who are going to struggle to understand the science, but who will be deciding the cases and passing the laws on AI. I'm old enough to remember when Ted Stevens called the internet "a system of tubes."
AI regulation can take many forms, using existing structures, or via new ones our governments create. The United States government (USG) has long favoured private ownership of corporations, including AI and tech companies, to encourage technology and investment. Contrast this with communist and socialist approaches, where state ownership is favoured. I think your proposal is the one favoured by the status quo, and I don't see the USG abandoning private ownership of corporations anytime soon, though the administration's recent acquisition of a stake in Intel is a new development and move towards a more socialist approach.
AI companies, as private corporations, are therefore already subject to all of the rules for corporations under US law, as well as subject to being sued in the courts, as is already happening to many AI companies. Our court system is the most powerful in the world and, in some ways, arguably the most powerful entity the world has ever seen. (The recent anti-trust litigation against Google is but one example.) Often, private companies in the US will also enact voluntary, internal standards as part of a good faith effort to demonstrate to the USG that they are capable of operating safely without intervention. One benefit of internal compliance systems is that they shift costs to the company and save the USG money.
There is also the question of whether or not the US requires bespoke regulation for AI corporations beyond what already exists. The EU has decided that it does via the AI Act; the US Congress is taking a wait-and-see approach. California is discussing this question right now. Because of the importance of California and the EU to the world economy, their rules will regulate US AI companies, and can therefore be viewed as a type of global regulation. We do not need to wait for the US federal government on this.
International cooperation on AI, whether through the UN, NATO or a bespoke treaty body, is a parallel option that can work in tandem with the regulatory frameworks discussed above, (with EU and CA law automatically setting global standards because of the size of the California and EU economies). Such global frameworks, like the UN, allow smaller countries to have more of a say and provide a forum to discuss how the system is working and pressure governments to take more concrete action. Some treaty mechanisms come with real enforcement mechanisms, others don't. Fitting AI into the existing system of treaties and treaty bodies is a big job that is currently under discussion at the UN, with UNESCO in some ways the most obvious choice. A bespoke treaty body is also an option.
Many, many authors point to the US economy juggernaut and rising living standards as proof of concept of the USG's light touch and collaborative approach when it comes to corporate regulation. The resiliency of the corporate framework shows in its use in everything from mining minerals, making chocolate bars and building AI. The good news is that the corporate structure we have can work in tandem with other options, like treaty bodies and the UN, so we don't need an either/or approach.