Snapshot of narratives and frames against regulating AI
4[anonymous]
3Seth Herd
2M. Y. Zuo
3quetzal_rainbow
3Dagon
3quetzal_rainbow
1M. Y. Zuo
1quetzal_rainbow
2Dagon
1M. Y. Zuo
1quetzal_rainbow
1M. Y. Zuo
13Dagon
8cfoster0
9Zac Hatfield-Dodds
2Nathan Helm-Burger
6Zac Hatfield-Dodds
5n0st
3Nathan Helm-Burger
New Answer
New Comment
3 Answers sorted by
40
Strong upvoted.
A more complex hypothesis is Meta doesn't actually love open source that much but has a sensible, self-interested strategy, aimed at a dystopian outcome.
I think that this particular dystopian outcome is Moloch, not "aimed" malevolence; I think that being "aimed at a dystopian outcome" is an oversimplification of the two-level game; complex internal conflict within the company, parallel to external conflict with other companies. For example, stronger AI and stronger brain-hacking/autoanalysis allows them to reduce the risk of users dropping below 2 hours of use per day (giving their platform a moat and securing the value of the company), while simultaneously reducing the risk of users spending 4+ hours per day which causes watchdog scrutiny, more AI means more degrees of freedom to reap the benefits of addiction with less of the unsightly bits.
I've previously described a hypothetical scenario where:
Facebook and the other 4 large tech companies (of whom Twitter/X is not yet a member due to vastly weaker data security and dominance by botnets) might be testing out their own pro-democracy anti-influence technologies and paradigms, akin to Twitter/X’s open-sourcing its algorithm, but behind closed doors due to the harsher infosec requirements that the big 5 tech companies face. Perhaps there are ideological splits among executives e.g. with some executives trying to find a solution to the influence problem because they’re worried about their children and grandchildren ending up as floor rags in a world ruined by mind control technology, and other executives nihilistically marching towards increasingly effective influence technologies so that they and their children personally have better odds of ending up on top instead of someone else.
Likewise, even if companies in both the US and China seem to currently eschew brain-hacking paradigms, they might reverse course at any time, especially if brain-hacking truly is the superior move for a company or government to make in the context of the current multimodal-based ML paradigm, especially in the current Cold-War style affairs for US-China.
Your and Gwern's "commoditize the complement" point is now a very helpful gear in my model, both for targeted influence tech and for modelling the US and Chinese tech industries more generally, thank you. Also, I had either forgotten or failed to realize that a thriving community of human creators allows for more intense influence strategies to be discovered by multi-armed bandit algorithms, rather than just being algorithmically bottlenecked or user/sensor data bottlenecked.
30
One possible framing is hubristic overconfidence vs. humble caution.
We aren't saying AGI will overthrow humanity soon if we're not careful; we're saying it could. Everyone saying that's ridiculous is essentially saying "hold my beer!" while attempting a stunt nobody has ever pulled off before. They could be right that it will be easy enough, they're smart enough, and they will know when the danger approaches. But they're gambling the future of all humanity on that confidence.
Experts have widely varying opinions on the dangers of AGI, so the humble belief is that we don't know what's possible, so should behave in accordance with accepting a very broad distribution of timelines, alignment difficulty, and levels of coordination.
That framing won't make the most concerned among us happy. It will result in people who aren't long-termists wanting to approach AGI fast enough to save themselves or their children from a painful death from natural causes. But it might be an acceptable compromise, while we gather and analyze more information.
21
How about “It’s not proven yet that vastly super-intelligent machines (i.e. >10x peak human intelligence) are even possible.” as a possible frame?
I can’t see a counterargument to it yet.
Even if we have only smartest-human-level models, you can spawn 100000 copies at 10x speed and organize them in the way "one model checks if output of other model displays cognitive biases" and get maybe not "design nanotech in 10 days" level but still something smarter than any organized group of humans.
3
Hmm. I've not seen any research about that possibility, which is obvious enough that I'd expect to see it if it were actually promising. And naively, it's not clear that you'd get more powerful results from using 1M times the compute this way, compared to more direct scaling.
I'd put that in the exact same bucket as "not known if it's even possible".
3
Such possibility is explored at least here: https://arxiv.org/abs/2305.17066 but that's not the point. The point is: even in hypothetical world where scaling laws and algorithmic progress hit the wall at smartest-human-level, you can do this and get an arbitrary level of intelligence. In real world, of course, there are better ways.
1
How do you know that's possible?
1
There are definitely enough matter on Earth to sustain additional 100k human brains with signal speed 1000m/s instead of 100m/s? I actually can't imagine how our understanding of physics should be wrong for it to not be possible.
2
I think you're using a different sense of the word "possible". In a simplified physics model, where mass and energy are easily transformed as needed, you can just wave your hands and say "there's plenty of mass to use for computronium". That's not the same as saying "there is an achievable causal path from what we experience now to the world described".
1
Did you misunderstand my question?
How does the total mass of the Earth or 'signal speed 1000m/s instead of 100m/s' demonstrate how you know?
1
The only reason why it can be impossible is if the amount of compute needed to run one smart-as-smartest human model is so huge that we need to literally disassemble Earth to run 100000 copies. It's quite unrealistic reason because similar amout of compute for actual humans fit an actual small cranium.
1
Why is the amount of matter in a human brain relevant?
An important sub-topic within "open source vs regulatory capture" is "there does not exist an authority that can legibly and correctly regulate AI".
Note that the outlook from MIRI folks appears to somewhat agree with this, that there does not exist an authority that can legibly and correctly regulate AI, except by stopping it entirely.
One possible hypothesis here is Meta just loves open source and wants everyone to flourish. ... A more complex hypothesis is Meta doesn't actually love open source that much but has a sensible, self-interested strategy
It's worth noting here that Meta is very careful never to describe Llama as open source, because they know perfectly well that it isn't. For example, here's video of Yan LeCun testifying under oath: "so first of all Llama system was not made open source ... we released it in a way that did not authorize commercial use, we kind of vetted the people who could download the model it was reserved to researchers and academics"
Worth noting about your note:
The distinction is irrelevant for misuse by bad actors, such as terrorist groups. The model weights were on the dark net very quickly after the supposedly controlled release.
Agreed: this kind of psudeo-openness has all of the downsides of releasing a dual-use capability, and we miss so many benefits from commercial use and innovation.
1I think the argument for the regulatory capture framing is basically that you have these things going on:
(1) big tech companies are spending extraordinary amounts of money, time, effort etc. on trying to capture the market of generative AI.
(2) big tech companies are arguing for regulations which on the face of them would be prohibitive to smaller players entering, because they would involve large fixed costs that would be trivial as a proportion of the available resources that these companies have, but would be prohibitive for smaller players.
Point (1) does not seem like something that an entity which is concerned about the fate of humanity would be doing.
I think the reality is obviously more complex, in that these big tech companies contain huge numbers of people who will have differing motivations at the conscious and subconscious level, so it's pretty difficult to answer what the motivation of the company as a whole "really is"... but I think if there is anything that it makes sense to say a corporation is "motivated to do", obviously earning profit is the most classic thing that corporations are "motivated to do"
Yes, I think the distinction between a company's goals/intents/behaviors in aggregate versus the intents and behaviors of individual employees is important. I know and trust individual people working at most of the major labs. That doesn't mean that I trust the lab as a whole will behave in complete harmony with the intent of those individual employees that I approve of.
Curated and popular this week