Questions for old LW members: how have discussions about AI changed compared to 10+ years ago?
8RHollerith
7Said Achmiz
3Viliam
3Brendan Long
5Expertium
3Viliam
2TAG
1Loki zen
1Expertium
1Loki zen
2Expertium
1Loki zen
New Comment
In 2015, I didn't write much about AI on Hacker News because even just explaining why it is dangerous will tend to spark enthusiasm for it in some people (people attracted to power, who notice that since it is dangerous, it must be powerful). These days, I don't let that consideration stop me from write about AI.
Yeah, 10 years ago it was: "we need smarter people, so that they can make the AI safe".
Today it is: "not enough time left to educate people, this is the crunch time".
- I think alignment is easier than I used to, since we can kind-of look into LLM's and find the concepts, which might let us figure out the knobs we need to turn even though we don't know what they are right now (i.e. weirdly enough, there might be a "lie to humans" button and we can just prevent the AI from pushing it). I still think it's unclear if we'll actually do the necessary research fast enough though. Alignment-by-default also seems more likely than I would have expected, although it does seem to be getting worse as we make LLM's larger. I'm not really sure how this has changed within the community since people who don't think AI is a problem don't really post about it.
- I think older posts about were mostly arguments about whether things could happen (could you make an oracle that's not an agent, could you keep the AI in a box, is AI even possible, etc.) and now that the AI doomers conclusively won all of those arguments, the discussions are more concrete (discussion of actually-existing AI features).
- It depends on what you mean by easier, but my timelines are shorter than they used to be, and I think most people's are.
- I'm definitely surprised that glorified decompression engines might be sufficient for AGI. The remaining problems don't really surprise me on top of knowing how they're trained[1]. I'm guessing the evolutionary AI people are feeling very vindicated though.
- ^
There's lots of coding training data and not very much training data for creating documents of a specific length. I think if we added a bunch of "Write ### words about X" training data the LLM's would suddenly be good at it.
1There's lots of coding training data and not very much training data for creating documents of a specific length. I think if we added a bunch of "Write ### words about X" training data the LLM's would suddenly be good at it.
My point was that it's surprising that AI is so bad at generalizing to tasks that it hasn't been trained on. I would've predicted that generalization would be much better (I also added a link to a post with more examples). This is also why I think creating AGI will be very hard, unless there will be a massive paradigm shift (some new NN architecture or a new way to train NNs).
EDIT: It's not "Gemini can't count how many words it has in its output" that surprises me, it's "Gemini can't count how many words it has in its output, given that it can code in Python and in a dozen other languages and can also do calculus".
"Alignment" is badly defined. It's ambiguous between "making AI safe, in anyway whatsoever" and "make Agentive AI safe by making it share human values" (As opposed to some other way, such as Control, or building only non Agentive AIs).
Alignment with "human values" is probably difficult, not least because "human value" isn't a well defined target. We can't define it as the output of CEV, because CEV isn't a method that's defined mechanically.
my take is that they haven't changed enough. People often still seem to be talking about agents and concepts that only make sense in the context of agents all the time - but LLMs aren't agents, they don't work that way. if often feels like the Agenda for the field got set 10+ years ago and now people are shaping the narrative around it regardless of how good a fit it is for the tech that actually came along.
But frontier labs are deliberately working on making LLMs more agentic. Why wouldn't they - AI that can do work autonomously is more economically valuable than a chatbot.
but they're not agents in the same way as the models in the thought experiments, even if they're more agentic. The base-level thing they do is not "optimise for a goal". We need to be thinking in terms of models that are shaped like the ones we actually have instead of holding on to old theories so hard we instantiate them in reality