Jack Clark Profile picture
Aug 6, 2022 81 tweets 13 min read Read on X
One like = one spicy take about AI policy.
A surprisingly large fraction of AI policy work at large technology companies is about doing 'follow the birdie' with government - getting them to look in one direction, and away from another area of tech progress
The vast majority of AI policy people I speak to seem to not be that interested in understanding the guts of the technology they're doing policy about
China has a much more well-developed AI policy approach than that in Europe and the United States. China is actually surprisingly good at regulating things around, say, synthetic media.
There is not some secret team working for a government in the West building incredibly large-scale general models. There are teams doing applied work in intelligence.
The real danger in Western AI policy isn't that AI is doing bad stuff, it's that governments are so unfathomably behind the frontier that they have no notion of _how_ to regulate, and it's unclear if they _can_
Many AI policy teams in industry are constructed as basic the second line of brand defense after the public relations team. A huge % of policy work is based around reacting to perceived optics problems, rather than real problems.
Many of the problems in AI policy stem from the fact that economy-of-scale capitalism is, by nature, anti-democratic, and capex-intensive AI is therefore anti-democratic. No one really wants to admit this. It's awkward to bring it up at parties (I am not fun at parties).
Lots of AI policy teams are disempowered because they have no direct technical execution ability - they need to internally horse-trade to get anything done, so they aren't able to do much original research, and mostly rebrand existing projects.
Most technical people think policy can't matter for AI, because of the aforementioned unfathomably-behind nature of most governments. A surprisingly large % of people who think this also think this isn't a problem.
A surprisingly large amount of AI policy is illegible, because mostly the PR-friendly stuff gets published, and many of the smartest people working in AI policy circulate all their stuff privately (this is a weird dynamic and probably a quirk/departure from norm)
Many of the immediate problems of AI (e.g, bias) are so widely talked about because they're at least somewhat tractable (you can make measures, you can assess, you can audit). Many of the longterm problems aren't discussed because no one has a clue what to do about them.
The notion of building 'general' and 'intelligent' things is broadly frowned on in most AI policy meetings. Many people have a prior that it's impossible for any machine learning-based system to be actually smart. These people also don't update in response to progress.
Many technologists (including myself) are genuinely nervous about the pace of progress. It's absolutely thrilling, but the fact it's progressing at like 1000X the rate of gov capacity building is genuine nightmare fuel.
The default outcome of current AI policy trends in the West is we all get to live in Libertarian Snowcrash wonderland where a small number of companies rewire the world. Everyone can see this train coming along and can't work out how to stop it.
Like 95% of the immediate problems of AI policy are just "who has power under capitalism", and you literally can't do anything about it. AI costs money. Companies have money. Therefore companies build AI. Most talk about democratization is PR-friendly bullshit that ignores this.
Some companies deliberately keep their AI policy teams AWAY from engineers. I regularly get emails from engineers at $bigtech asking me to INTRO THEM to their own policy teams, or give them advice on how to raise policy issues with them.
Sometimes, bigtech companies seem to go completely batshit about some AI policy issue, and 90% of the time it's because some internal group has figured out a way to run an internal successful political campaign and the resulting policy moves are about hiring retention.
Some people who work on frontier AI policy think a legitimate goal of AI policy should be to ensure governments (especially US government) has almost no understanding of rate of progress at the frontier, thinking it safer for companies to rambo this solo (I disagree with this).
It's functionally impossible to talk about the weird (and legitimate) problems of AI alignment in public/broad forums (e.g, this twitter thread). It is like signing up to be pelted with rotten vegetables, or called a bigot. This makes it hard to discuss these issues in public.
AI really is going to change the world. Things are going to get 100-1000X cheaper and more efficient. This is mostly great. However, historically, when you make stuff 100X-1000X cheaper, you upend the geopolitical order. This time probably won't be different.
People wildly underestimate how much influence individuals can have in policy. I've had a decent amount of impact by just turning up and working on the same core issues (measurement and monitoring) for multiple years. This is fun, but also scares the shit out of me.
AI policy is anti-democratic for the same reasons as large-scale AI being anti-democratic - companies have money, so they can build teams to turn up at meetings all the time and slowly move the overton window. It's hard to do this if it's not your dayjob.
Lots of the seemingly most robust solutions for reducing AI risk require the following things to happen: full information sharing on capabilities between US and China and full monitoring of software being run on all computers everywhere all the time. Pretty hard to do!
It's likely that companies are one of the most effective ways to build decent AI systems - companies have money, can move quickly, and have fewer stakeholders than governments. This is a societal failing and many problems in AI deployment stem from this basic fact.
Most technologist feel like they can do anything wrt AI because governments (in West) have shown pretty much zero interest in regulating AI, beyond punishing infractions in a small number of products. Many orgs do skeezy shit under the radar and gamble no one will notice.
Discussions about AGI tend to be pointless as no one has a precise definition of AGI, and most people have radically different definitions. In many ways, AGI feels more like a shibboleth used to understand if someone is in- or out-group wrt some issues.
The concept of 'information hazards' regularly ties up some of the smartest people and causes them to become extremely unproductive and afraid to talk or think about certain ideas. It's a bit of a mind virus.
At the same time, there are certain insights which can seem really frightening and may actually be genuine information hazards, and it's very hard to understand when you're being appropriately paranoid, and when you're being crazy (see above).
It's very hard to bring the various members of the AI world together around one table, because some people who work on longterm/AGI-style policy tend to ignore, minimize, or just not consider the immediate problems of AI deployment/harms. V alienating.
Most people working on AI massively discount how big of a deal human culture is for the tech development story. They are aware the world is full of growing economic inequality, yet are very surprised when people don't welcome new inequality-increasing capabilities with joy.
People don't take guillotines seriously. Historically, when a tiny group gains a huge amount of power and makes life-altering decisions for a vast number of people, the minority gets actually, for real, killed. People feel like this can't happen anymore.
IP and antitrust laws actively disincentivize companies from coordinating on socially-useful joint projects. The system we're in has counter-incentives for cooperation.
Lots of AI research papers are missing really important, seemingly inconsequential details, that are actually fundamental to how the thing works. Most people in most labs spend time spotting the 'hidden facts' in papers from other labs. It's a very weird game to play.
Many people developing advanced AI systems feel they're in a race with one another. Half of these people are desperately trying to change the race dynamics to stop the race. Some people are just privately trying to win.
In AI, like in any field, most of the people who hold power are people who have been very good at winning a bunch of races. It's hard for these people to not want to race and they privately think they should win the race.
Doing interdisciplinary work in AI policy is incredibly difficult. Corporate culture pushes against it, and to bring on actual subject-matter experts you need to invest tons of $$$ in systems to make your infrastructure easier to deal with. It requires huge, sustained effort.
Most universities are wildly behind the private labs in terms of infrastructure and scale. Huge chunks of research have, effectively, become private endeavors due to the cost of scale. This is also making universities MORE dependent on corporations (e.g for API access).
Many people who stay in academic specialize in types of research that doesn't require huge experimental infrastructure (e.g, multiple-thousand GPU clusters). This means the next generation of technologists are even more likely to go to private sector to study at-scale problems.
Scale is genuinely important for capabilities. You literally can't study some problems if you futz around with small models. This also means people who only deal with small models have broken assumptions about how the tech behaves at scale.
At the different large-scale labs (where large-scale = multiple thousands of GPUs), there are different opinions among leadership on how important safety is. Some people care about safety a lot, some people barely care about it. If safety issues turn out to be real, uh oh!
Working in AI right now feels like how I imagine it was to be a housing-debt nerd in the run-up to the global financial crisis. You can sense that weird stuff is happening in the large, complicated underbelly of the tech ecosystem.
AI policy can make you feel completely insane because you will find yourself repeating the same basic points (academia is losing to industry, government capacity is atrophying) and everyone will agree with you and nothing will happen for years.
AI policy can make you feel completely exhilarated because sometimes you meet people who have a) vision, b) power, and c) an ability to stay focused. You can do tremendous, impactful work if you find these people, and finding them is 50% of my job. (This thread is a beacon!)
'Show, don't tell' is real. If you want to get an idea across, demo a real system, live. This will get you 100X the impact of some PR-fied memo. Few people seem to do live demos because lack of choreography freaks them out. But policymakers hate dog and pony shows. What gives?
AI development is a genuine competition among nations. AI is crucial to future economic and national security. Lots of people who make safety/risk-focused arguments to policymakers don't acknowledge this prior and as a consequence their arguments aren't listened to.
If you have access to decent compute, then you get to see the sorts of models that will be everywhere in 3-5 years, and this gives you a crazy information asymmetry advantage relative to everyone without a big computer.
China is deriving a real, strategic advantage by folding in large-scale surveillance+AI deals as part of 'One Belt One Road' investment schemes worldwide. China is innovating on architectures for political control.
AI may be one of the key ingredients to maintaining political stability in the future. If Xi is able to retain control in China, the surveillance capabilities of AI will partially be why. This has vast and dire implications for the world - countries copy what works.
The inequality and social discord in the West may ultimately prevent us from capturing many of the advantages AI could be giving to society - people are outright rejecting AI in many contexts due to the capitalist form of development.
Policy is permissionless - companies drill employees to not talk to policymakers and only let those talks happen through gov affairs teams and choreographed meetings. This isn't a law, it's just brainwashing. Engineers should talk directly to policy people.
One of the most effective ways to advocate for stuff in policy is to quantify it. The reason 30% of my life is spent turning data points from arXiv into graphs is that this is the best way to alter policy - create facts, then push them into the discourse.
Correspondingly, one of the ways to be least effective in policy is to base your position around qualitative interpretations of the domain, and to mostly use rhetoric to make your points - this makes people switch off.
Most policy forums involve people giving canned statements of their positions, and everyone thanks eachother for giving their positions, then you agree it was good to have a diverse set of perspectives, then the event ends. Huge waste of everyone's time.
To get stuff done in policy you have to be wildly specific. CERN for AI? Cute idea. Now tell me about precise funding mechanisms, agency ownership, plan for funding over long-term. If you don't do the details, you don't get stuff done.
Policy is a gigantic random number generator - some random event might trigger some politician to have a deep opinion about an aspect of AI, after which they don't update further. This can brick long-term projects randomly (very relaxing).
It is extremely bad that most of the people with significant political power in the US and other govs are 50+. They do not conceptualize this stuff the same way younger people do, and therefore don't recognize as legitimate a bunch of fears and opportunities.
There are endless debates in policy forums about how important it is to focus on symbolic systems as well as deep learning systems. It's frequently unclear what people mean by symbolic and there aren't many benchmarks (any?) where symbolic systems show up. Dark matter.
Richard Sutton's The Bitter Lesson is one of the best articulations of why huge chunks of research are destined to be irrelevant as a consequence of scale. This makes people super mad, but also seems like a real phenomenon.
AI is so strategic to so many companies that it has altered the dynamics of semiconductor development. Because chips take years to develop, we should expect drastic improvements in AI efficiency in the future, which has big implications on diffusion of capabilities.
Attempts to control AI (e.g content filters) directly invite a counter-response. E.g, Dall-E vs #stablediffusion. It's not clear that the control methods individual companies use help relative to the bad ecosystem reactions to these control methods. (worth trying tho)
Most policymakers presume things exist which don't actually exist - like the ability to measure or evaluate a system accurately for fairness. Regulations are being written where no technology today exists that can be used to enforce that regulation.
Governments tend to literally not know how much money they spend on AI. Getting accurate funding data is extremely difficult, even for civil/public areas (been working on this for years via @indexingai ). You can't manage what you can't measure.
Code models are going to change some of the game theory of cyber offense/defense dynamics and the capabilities are going to cause real problems.
The fact it's hard to a priori predict capabilities in larger-scale models means a natural dividend of model development is scary/unwanted capabilities that need be somehow found after you train it and before you deploy it. Incredibly difficult. Possibly intractable.
Norms and best practices only work on people who have an incentive to adopt them (e.g, companies to minimize PR/policy risks). The hard problem is coming up with enforcement mechanisms that can influence the people who don't care about norms and best practices.
Pretty much everyone who works on AI thinks that they're 'one of the good people'. Statistically, this is unlikely to be the case.
Recommendation models will, by design, eventually succeed at modelling an individual for a given predictive task by using information efficiently - aka, v few bits to calibrate. Social media companies are building models that will ultimately be superhuman at modelling people.
Proactively giving up power is one of the hardest things for people to do. Giving up power and money is even harder. AI orgs are rapidly gathering power and money and it's not clear they have right incentives to willfully shed their own power. This sets us up for racing.
It's not clear what the optimal form of large-scale AI deployment should look like - private sector actors? Government actors? Public-private hybrids? There isn't really a single architecture people are thinking about for Really Big Models and this is an open problem.
For years, people built models then built safety tooling around them. People are now directly injecting safety into models via reinforcement learning for human feedback. Everyone is DIY'ing these values, so the values are subjective via the tastes of people within each org.
Building third-party AI auditing organizations via gov is challenging due to needing to wire-in funding. However, doing this in private sector requires CEO with vision most people who could do this are either not willing to go FT on it, or feel time is better spent at AI cos.
A lot of the mismatch between AI impact and societal benefit is due to a lack of ad-hoc technical experimentation capacity on part of govs/civil society wrt frontier models. They may have access to models, but you need to invest people time to help them explore. Hire a Virgil.
'street-level AI ' has already begun to change the nature of military conflict. It started in ~2015-ish, but in Ukraine this year we saw troops pair $10-20k drones with 1950s grenades and 3D-printed aero-fins, targeted via vision models trained to spot soldiers in camo.
Seriously, the above point is worth belaboring - for certain types.of conflict cheap robots and a bit of AI has drastically reduced cost and increased time-efficiency. My 5min $20k drone and $500 grenade and $20 fin and 2-person team destroy your $2-4m tank and associated people
Malware is bad now but will be extremely bad in the future due to intersection of RL + code models + ransomware economic incentives. That train is probably 1-2 years away based on lag of open source replication of existing private models, but it's on the tracks.
Deepfakes have mostly been a porn thing, but we've had a few cases in politics (eg Gabon a few years ago, and some stuff in Ukraine). Deepfakes on smartphones is gonna be a thing soon - models appear and get miniaturized and open-sourced then made easy to use. Proliferation++
Surveillance has already been radically changed by AI. Even public open source models (e.g YOLO) are perfectly usable, downloadable, and free. This means you can track people. But that's not the interesting part...
If you see a person a bit you can learn to identify them _even with some outfit change_ and then you can pick them up from other cameras in diff lighting and angles. The models used for this stuff are getting better at a n-2yr basis vs big models due to needing 30fps+ inference.
Trivia: the original developer of YOLO ceased developing it after V3 [2018: arxiv.org/abs/1804.02767] due to reasons outlined below - - yolo (now v7) has subsequently been pushed forward by Taiwan/Russia/China. Some things are kind of locked-in... [Amazing paper overall btw!] Image
The fact large-scale models display qualitatively new capabilities is both extremely cool and scary. Since we can't predict capabilities ahead of time, there's a plausible future where a model appears and creatively lies/outsmarts us in a way we can't effectively measure.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Jack Clark

Jack Clark Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @jackclarkSF

Jun 25, 2023
Will write something longer, but if best ideas for AI policy involve depriving people of the 'means of production' of AI (e.g H100s), then you don't have a hugely viable policy. (I 100% am not criticizing @Simeon_Cps here; his tweet highlights how difficult the situation is).
I gave a slide preso back in fall of 2022 along these lines. Including some slides here. The gist of it is if you basically go after compute in the wrong ways you annoy a huge amount of people and you guarantee pushback and differential tech development.



Since I gave that presentation we've seen:
- people build chatGPT clone models by training on chatGPT outputs
- low-cost finetuning like LORA / QLORA etc
- increase in number of actors trying to do open/decentralized model development (e.g )
- etctogether.xyz
Read 22 tweets
Feb 12, 2023
A mental model I have of AI is it was roughly ~linear progress from 1960s-2010, then exponential 2010-2020s, then has started to display 'compounding exponential' properties in 2021/22 onwards. In other words, next few years will yield progress that intuitively feels nuts.
There's pretty good evidence for the extreme part of my claim - recently, language models got good enough we can build new datasets out of LM outputs and train LMs on them and get better performance rather than worse performance. E.g, this Google paper: arxiv.org/abs/2210.11610
We can also train these models to improve their capabilities through use of tools (e.g, calculators, QA systems), as in the just-came-out 'Toolformer' paper arxiv.org/abs/2302.04761 .
Another fav of mine= this wild paper where they staple MuJoCo to an LM arxiv.org/abs/2210.05359
Read 5 tweets
Jan 29, 2023
Modern AI development highlights the tragedy of letting the private sector lead AI invention - the future is here but it's mostly inaccessible due to corporations afraid of PR&Policy risks. (This thought sparked by Google not releasing its music models, but trend is general). The 21st century is being d...
There will of course be exceptions and some companies will release stuff. But this isn't going to get us many of the benefits of the magic of contemporary AI. We're surrendering our own culture and our identity to the logic of markets. I am aghast at this. And you should be too.
I've written a lot about this in Import AI 316 (which comes out tomorrow), as well as a short story about what commercially-led data gathering leads us to. The gates of heaven and hell are open, to paraphrase someone else who works in AI.
Read 5 tweets
Nov 5, 2022
If you want a visceral sense of how different development practices and strategies can lead to radically different performance, compare and contrast performance of the BLOOM and GLM-130B LLMs.

huggingface.co/bigscience/blo…

huggingface.co/spaces/THUDM/G…
Feels kind of meaningful that an academic group at Tsinghua University (GLM-130B) made a substantially better model than a giant multi-hundred person development project (BLOOM).
I thought a diff could be data, but doesn't seem like it - BLOOM was trained on 350 billion tokens and GLM-130B on 400 billion tokens (more tokens = better). Not a substantial enough gulf to solely explain the perf differences
Read 6 tweets
Jul 17, 2022
In late May, I had back spasms for 24 hours, then couldn't walk for a week, then spent a month+ recovering. It was one of the worst experiences of my life and I'm glad I seem to now be mostly recovered. Here are some things that happened that seemed notable during that time:
Being genuinely disabled is really frightening. I couldn't feed myself for the first few days. Any time I needed to go to bathroom I had to do it predominantly using upper body strength. Sometimes I'd fall over and not be able to get up for hours.
I had back spasms that were so bad they'd physically bounce me out of bed. I became acutely paranoid about sharp things in my room as there was a real risk I'd hit my head on them when spasming. The spasms were easily a 10 on the 0-10 pain scale.
Read 12 tweets
Jul 1, 2022
As someone who has spent easily half a decade staring at AI arXiv each week and trying to articulate rate of progress, I still don't think people understand how rapidly the field is advancing. Benchmarks are becoming saturated at ever increasing rates.
Here's MINERVA which smashes prior math benchmarks by double digit percentage point improvements
Here's a system that beats Stratego, a game with complexity far, far higher than Go.
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(