This is a quick write-up of part 2 of my VAISU talk. See also part 1, and longer post.


td;lr  A rough framework for understanding the ways AGI gets selected for, 
and the ways communities can constrain that selection by acting now.  

Selection toward AGI

Types of feedback loops through which AGI^[1] functionality is gradually selected:

1. Machine learning    
   Code internally optimised given data for continued capacity to optimise for next data inputs (toward instrumental convergence).
    
2. Human tinkering 
   Workers tinkering with code and hardware for whatever works, as new tools used for tinkering... (toward larger neural networks and robotics that somehow work).
    
3. Institutions competing
   Corporate uses of AI that extract more profit/social influence/resources, which are reinvested into more machinery, allowing more uses… (toward multi-polar traps).
    
4. Artificial selection
   Components' external effects selected for feeding back into their continued increased existence (toward substrate-needs convergence).
    

These loops are mutually reinforcing, in where and at what scale, they select for AGI.^[1]

• 1. (within hardware) and 2. (within institutions) are about internal selection.

• 3. (via markets/social hierarchies/infrastructure) and 3. (via environmental effects) are about external selection.

Prerequisites for AGI

Prerequisites^[2] for that selection to occur:

1. Data (laundering)
   pirated from original authors, and surveyed from communities and their spaces.
   eg. Text and images scraped from creatives. Exchanges scraped from citizens.
   Data from data workers are underpaid if paid at all.
    
2. Workers (exploitation)
   misled to dedicate their talent, and/or eventually replaced.
   eg. Talented ML researchers are recruited with tech utopia claims. Software engineers are hired to set up infrastructure, and then fired. A much larger number of data workers and creatives are also misled to take on jobs for training of ML models.
    
3. Misuses (of AI services)
   lobbied for by corporations to gain profit and power. 
   eg. General-Purpose AI is a marketing term implying that a given model can be and should be used for any purpose. While profitable for Big Tech, this would violate basic system-safety practices in eg. medical, automotive and industrial engineering. 
    
4. Compute (pollution)
   polluting, energy-sucking hardware infrastructure.
   eg. Of the mining, production, supply, and operation chains for chipsets.

AI companies compete on, and are bottlenecked by, the availability of each prerequisite. 
The more we can restrict the data, workers, misuses, and compute available to AI companies (and other groups), the more we restrict paths toward AGI.

This is in rough order from least to most diffuse harms:

1. Personal:  Taking someone's own data, for commercial ends, without their consent.
2. Institutional: Misleading employees to contribute to developing harmful products.
3. Societal:  Destabilising society with poorly designed spaghetti code.
4. Ecological:  Polluting the planetary environment.

There is ambiguity across these categories. 
Eg. Underpaying digital freelancers and taking their unique works to train AI involve both directly personal and institution-wide harms. And while the misuse of autonomous network-centric weapons by national militaries is harmful society-wide, the data surveillance of individual citizens targeted by drones is personal.

Less diffuse harms are easier to trace and for individuals affected to act upon.

• The copying of data can in principle be traced easily. If an engineer at a whistleblows and discloses harmful training datasets, they can be linked to specific authors/citizens affected.
• Very diffuse harms like leakage/emission of chemicals resulting from hardware mining, production and operation are much harder to trace and link to harmed persons. Then you get into into climate-change-like coordination problems.

What follows roughly is that we can make more traction by starting at 1. 
That is, first help scale projects to inform people how they are personally harmed, and enable them to take targeted legal actions. People informed of harms directed at them are usually more motivated to act effectively to restrict those harms, than people informed of abstract distributed harms (climate change activism, notwithstanding).

Projects to restrict increasingly diffuse harms of 2, 3, and 4 take more time to coordinate.
But to comprehensively restrict the selective pressures, those projects are needed too.

Eventually, such parallel actions could converge on converge on robust bans:

1. Data restrictions
   Ban input bandwidth above certain limits.
2. Human-in-the-loop restrictions:  
   Ban having input, processing, and output domains be the same.
3. Model use restrictions:
   Ban multiple-domain ML models.
   Ban output bandwidth/intensity above certain limits.
4. Compute restrictions
   Ban compute processing bandwidth beyond certain limits.

More thoughts from my research mentor, Forrest Landry:
"Models need to be domain restricted. Ie, only train on one domain of interaction, and then only allow one domain of input and output – preferably different ones that are NOT in any way feedback coupled to one another through any even indirect routing of just the physical universe. Ie, that any different domain indirection coupling must and can only occur through human intermediation. Ie, humans provide inputs, and the machine outputs in a different domain of action."

Movements resisting AI

Broader movements that are acting to restrict each prerequisite:

1. Data rights
   Data auditors uncovering racially biased, copyrighted, and pornographic materials. 
   Privacy regulators restricting/suing OpenAI, Google, Facebook, and Amazon.
   Creatives and privacy orgs lobbying, and suing OpenAI, Google, and Facebook.
   Data workers organising and filing lawsuits.
   ...
2. Employee activism
   Workers unionising and organising strikes against AI replacing their jobs.
   AI researchers quitting, and potentially whistleblowing.
   ... 
3. System safety
   AI ethics researchers advocating against shoddy harmful design practices.
   Interpretability researchers testing for misusable deceptive functionality.
   Adversarial and cybersecurity researchers reporting adversarial backdoors.
   Safety engineers advising on porting over auditing practices to an "AI FDA".
   Military vets and journalists investigating corruption and the Kill Cloud.
   ...
4. Environmentalism
   Climate researchers and reporters reporting on pollution across supply chains.
   Local residents protesting against data centers.
   Potentially, mass lawsuits.
    

What are those movements resisting AI each bottlenecked by?

Funding. 
No-strings attached.

1. ^{^}

   A more precise way of redefining AGI is 'self-sufficient learning machinery'.

2. ^{^}

   The selection mechanisms don't correspond one-on-one with the prerequisites.